Siamese Neural Networks: An Overview

Human papillomavirus (HPV) prediction for oropharyngeal cancer based on CT by using off-the-shelf features: A dual-dataset study

BACKGROUND

This study aims to develop a novel predictive model for determining human papillomavirus (HPV) presence in oropharyngeal cancer using computed tomography (CT). Current image-based HPV prediction methods are hindered by high computational demands or suboptimal performance.

METHODS

To address these issues, we propose a methodology that employs a Siamese Neural Network architecture, integrating multi-modality off-the-shelf features-handcrafted features and 3D deep features-to enhance the representation of information. We assessed the incremental benefit of combining 3D deep features from various networks and introduced manufacturer normalization. Our method was also designed for computational efficiency, utilizing transfer learning and allowing for model execution on a single-CPU platform. A substantial dataset comprising 1453 valid samples was used as internal validation, a separate independent dataset for external validation.

RESULTS

Our proposed model achieved superior performance compared to other methods, with an average area under the receiver operating characteristic curve (AUC) of 0.791 [95% (confidence interval, CI), 0.781-0.809], an average recall of 0.827 [95% CI, 0.798-0.858], and an average accuracy of 0.741 [95% CI, 0.730-0.752], indicating promise for clinical application. In the external validation, proposed method attained an AUC of 0.581 [95% CI, 0.560-0.603] and same network architecture with pure deep features achieved an AUC of 0.700 [95% CI, 0.682-0.717]. An ablation study confirmed the effectiveness of incorporating manufacturer normalization and the synergistic effect of combining different feature sets.

CONCLUSION

Overall, our proposed model not only outperforms existing counterparts for HPV status prediction but is also computationally accessible for use on a single-CPU platform, which reduces resource requirements and enhances clinical usability.

CA-SQBG: Cross-attention guided Siamese quantum BiGRU for drug-drug interaction extraction

Accurate and efficient drug-drug interaction extraction (DDIE) from the medical corpus is essential for pharmacovigilance, drug therapy and drug development. To solve the problems of unbalance dataset and lack of accurate manual annotations in DDIE, a cross-attention guided Siamese quantum BiGRU (CA-SQBG) is constructed to improve feature representation learning ability for DDIE. It mainly consists of two quantum BiGRUs (QBiGRUs) and a cross-attention, where two QBiGRUs are Siamese implemented in a variational quantum environment to learn the contextual semantic feature representation of drug pairs, cross-attention is employed to learn mutual information from the Siamese QBiGRUs, which in turn allows the two modules to extract DDI more collaboratively. Unlike BiGRU, Siamese QBiGRUs uses internal and external dependencies in quaternion algebra to map DDI correlations within and between multidimensional features, whereas BiGRU can only capture dependencies within sequences. CA-SQBG is evaluated on the DDIExtraction2013 dataset, and the results demonstrate that it can effectively capture the inter- and intra-dependencies within multimodal features with few parameters, using a small number of training samples, and is superior to the most advanced DDIE methods. CA-SQBG offers potential applications for quantum computing and Siamese networks in the field of DDIE. Code is available on https://github.com/xaycq/CA-SQBG.

Merging metabolic modeling and imaging for screening therapeutic targets in colorectal cancer

Cancer-associated fibroblasts (CAFs) play a key role in metabolic reprogramming and are well-established contributors to drug resistance in colorectal cancer (CRC). To exploit this metabolic crosstalk, we integrated a systems biology approach that identified key metabolic targets in a data-driven method and validated them experimentally. This process involved a novel machine learning-based method to computationally screen, in a high-throughput manner, the effects of enzyme perturbations predicted by a computational model of CRC metabolism. This approach reveals the network-wide effects of metabolic perturbations. Our results highlighted hexokinase (HK) as a crucial target, which subsequently became our focus for experimental validation using patient-derived tumor organoids (PDTOs). Through metabolic imaging and viability assays, we found that PDTOs cultured in CAF-conditioned media exhibited increased sensitivity to HK inhibition, confirming the model predictions. Our approach emphasizes the critical role of integrating computational and experimental techniques in exploring and exploiting CRC-CAF crosstalk.

Transductive Few-shot Learning With Enhanced Spectral-Spatial Embedding for Hyperspectral Image Classification

Few-shot learning (FSL) has been rapidly developed in the hyperspectral image (HSI) classification, potentially eliminating time-consuming and costly labeled data acquisition requirements. Effective feature embedding is empirically significant in FSL methods, which is still challenging for the HSI with rich spectral-spatial information. In addition, compared with inductive FSL, transductive models typically perform better as they explicitly leverage the statistics in the query set. To this end, we devise a transductive FSL framework with enhanced spectral-spatial embedding (TEFSL) to fully exploit the limited prior information available. First, to improve the informative features and suppress the redundant ones contained in the HSI, we devise an attentive feature embedding network (AFEN) comprising a channel calibration module (CCM). Next, a meta-feature interaction module (MFIM) is designed to optimize the support and query features by learning adaptive co-attention using convolutional filters. During inference, we propose an iterative graph-based prototype refinement scheme (iGPRS) to achieve test-time adaptation, making the class centers more representative in a transductive learning manner. Extensive experimental results on four standard benchmarks demonstrate the superiority of our model with various handfuls (i.e., from 1 to 5) labeled samples. The code will be available online at https://github.com/B-Xi/TIP_2025_TEFSL.

Enhancing semantical text understanding with fine-tuned large language models: A case study on Quora Question Pair duplicate identification

Semantical text understanding holds significant importance in natural language processing (NLP). Numerous datasets, such as Quora Question Pairs (QQP), have been devised for this purpose. In our previous study, we developed a Siamese Convolutional Neural Network (S-CNN) that achieved an F1 score of 82.02% (95% C.I.: 81.83%-82.20%). Given the growing attention toward large language models (LLMs) like ChatGPT, we aimed to explore their effectiveness in text similarity tasks. In this research, we leveraged 5 pretrained LLMs, conducted various fine-tuning approaches (prompt engineering, n-shot learning, and supervised learning using the low-rank adaptation [LoRA]), and compared their performance using F1 score. To ensure a fair comparison, we followed our previous study's design and dataset by employing a 10-fold cross-validation for supervised model training and evaluation. Additionally, we conducted a secondary study by introducing a recent larger LLM with 70B parameters and comparing it with the 7B model using the GLUE benchmark, and both models were finetuned with the corpus. The fine-tuned LLaMA model with 7B parameters (qLLaMA_LoRA-7B) using 100,000 QQP corpus yielded the best results, achieving an F1 score of 84.9% (95% C.I.: 84.13%-85.67%), which outperformed the Alpaca_LoRA-65B (finetuned based on LLaMA-65B) (F1: 64.98% [64.72%-65.25%]; P<0.01) and had a 3% improvement compared to our previously published best model, S-CNN. The finetuned LLaMA3.1-70B (qLLaMA3.1_LoRA-70B) with 70B parameters (F1: 74.4%) outperformed the qLLaMA_LoRA-7B (F1: 71.9%) using the GLUE benchmark. The study demonstrated an effective LLM finetuning framework, which highlights the importance of finetuning LLMs for improved performance. Our task-specific supervised finetuning demonstrated improved LLM performance compared to larger pretrained models with or without n-shot learning; moreover, finetuning a larger LLM further improved performance compared to finetuning a smaller LLM. Our LLM-based finetuning framework may potentially improve various document similarity tasks, such as matching resumes with job descriptions, recommending subject-matter experts, or identifying potential reviewers for grant proposals or manuscript submissions.

Rapid and Precise Topological Comparison with Merge Tree Neural Networks

Merge trees are a valuable tool in the scientific visualization of scalar fields; however, current methods for merge tree comparisons are computationally expensive, primarily due to the exhaustive matching between tree nodes. To address this challenge, we introduce the Merge Tree Neural Network (MTNN), a learned neural network model designed for merge tree comparison. The MTNN enables rapid and high-quality similarity computation. We first demonstrate how to train graph neural networks, which emerged as effective encoders for graphs, in order to produce embeddings of merge trees in vector spaces for efficient similarity comparison. Next, we formulate the novel MTNN model that further improves the similarity comparisons by integrating the tree and node embeddings with a new topological attention mechanism. We demonstrate the effectiveness of our model on real-world data in different domains and examine our model's generalizability across various datasets. Our experimental analysis demonstrates our approach's superiority in accuracy and efficiency. In particular, we speed up the prior state-of-the-art by more than 100× on the benchmark datasets while maintaining an error rate below 0.1%.

Intelligent open-set MIMO recognition in OWC using a Siamese neural network

Multiple-input multiple-output (MIMO) technology, a core component of 6G, has been widely adopted in optical wireless communication (OWC) systems. Accurate recognition of different MIMO types is essential for MIMO selection and demodulation. In this Letter, we propose an open-set MIMO recognition method for OWC systems using a Siamese neural network (SNN). Simulation results show that the SNN significantly outperforms other recognition approaches, including convolutional neural networks (CNNs) and traditional machine learning techniques. For SNN-based recognition, over 90% accuracy is achieved with training based on only nine fixed sampling points in both 2 × 2 and 4 × 4 MIMO-OWC systems.

Merging Metabolic Modeling and Imaging for Screening Therapeutic Targets in Colorectal Cancer

Cancer-associated fibroblasts (CAFs) play a key role in metabolic reprogramming and are well-established contributors to drug resistance in colorectal cancer (CRC). To exploit this metabolic crosstalk, we integrated a systems biology approach that identified key metabolic targets in a data-driven method and validated them experimentally. This process involved a novel machine learning-based method to computationally screen, in a high-throughput manner, the effects of enzyme perturbations predicted by a computational model of CRC metabolism. This approach reveals the network-wide effects of metabolic perturbations. Our results highlighted hexokinase (HK) as a crucial target, which subsequently became our focus for experimental validation using patient-derived tumor organoids (PDTOs). Through metabolic imaging and viability assays, we found that PDTOs cultured in CAF-conditioned media exhibited increased sensitivity to HK inhibition, confirming the model predictions. Our approach emphasizes the critical role of integrating computational and experimental techniques in exploring and exploiting CRC-CAF crosstalk.

SiCRNN: A Siamese Approach for Sleep Apnea Identification via Tracheal Microphone Signals

Sleep apnea syndrome (SAS) affects about 3-7% of the global population, but is often undiagnosed. It involves pauses in breathing during sleep, for at least 10 s, due to partial or total airway blockage. The current gold standard for diagnosing SAS is polysomnography (PSG), an intrusive procedure that depends on subjective assessment by expert clinicians. To address the limitations of PSG, we propose a decision support system, which uses a tracheal microphone for data collection and a deep learning (DL) approach-namely SiCRNN-to detect apnea events during overnight sleep recordings. Our proposed SiCRNN processes Mel spectrograms using a Siamese approach, integrating a convolutional neural network (CNN) backbone and a bidirectional gated recurrent unit (GRU). The final detection of apnea events is performed using an unsupervised clustering algorithm, specifically k-means. Multiple experimental runs were carried out to determine the optimal network configuration and the most suitable type and frequency range for the input data. Tests with data from eight patients showed that our method can achieve a Recall score of up to 95% for apnea events. We also compared the proposed approach to a fully convolutional baseline, recently introduced in the literature, highlighting the effectiveness of the Siamese training paradigm in improving the identification of SAS.

ExCS: accelerating code search with code expansion

Efficiently searching and reusing code from expansive codebases is pivotal for enhancing developers' productivity. In recent times, the emergence of deep learning-driven neural ranking models, characterized by their vast dimensions and intricate interaction mechanisms, has been noteworthy. Yet, these models, in real-world scenarios, pose computational challenges due to their high dimensionality. Moreover, models rooted in interaction necessitate querying every piece of code within a voluminous corpus. While these methodologies offer superior accuracy, their online retrieval process is considerably more time-consuming compared to traditional Information Retrieval (IR) techniques. Addressing this, we introduce "ExCS", an innovative code search tool designed to expedite the code search process without compromising on accuracy. ExCS innovatively employs code expansion in its offline phase, leveraging predictions on potential queries for specific codes, thereby enriching the code's semantic depth. During online retrieval, ExCS prioritizes IR-based methods to pinpoint a concise set of persuasive candidates. Our evaluations, conducted on the Java dataset from CodeSearchNet, reveal that ExCS achieves a remarkable 90% reduction in retrieval duration while maintaining an impressive 99% retrieval accuracy.

Automated mapping of electronic data capture fields to SDTM

OBJECTIVE

The goal of this work is to reduce the amount of manual work required to go from data capture to regulatory submission. It will be shown that the use of Siamese networks will allow for the generation of embeddings that can be used by traditional machine learning classifiers to perform the classification at much higher levels of accuracy than standard approaches.

METHODS

Siamese networks are a method for training data embeddings such that data within the same class are closer with respect to a given distance metric than they are to data points in another class. Because they are designed to learn similarity within pairs of data points, they work well in situations where the number of classes is relatively large compared to the number of training samples. In this work, we will show that embeddings generated via a Siamese network from metadata associated with electronic data capture forms can be used to predict the associated SDTM field.

RESULTS

With a relatively simple network coupled with a basic classification algorithm, the proposed method can achieve accuracies greater than 90%, which is significantly higher than what has been achieved with traditional methods, with many of the inaccurate mappings due to a lack of training data. In many cases, there is a 15% increase in accuracy vs. more traditional methods.

CONCLUSION

Leveraging Siamese networks, it is possible to generate embeddings that efficiently represent data fields in a lower dimensional space. This allows the creation of a system that can automatically map between data schemas at high levels of accuracy. Such systems represent the first step in automating one of the many labor-intensive data management tasks associated with clinical trials.

SWFormer: Stochastic Windows Convolutional Transformer for Hybrid Modality Hyperspectral Classification

Joint classification of hyperspectral images with hybrid modality can significantly enhance interpretation potentials, particularly when elevation information from the LiDAR sensor is integrated for outstanding performance. Recently, the transformer architecture was introduced to the HSI and LiDAR classification task, which has been verified as highly efficient. However, the existing naive transformer architectures suffer from two main drawbacks: 1) Inadequacy extraction for local spatial information and multi-scale information from HSI simultaneously. 2) The matrix calculation in the transformer consumes vast amounts of computing power. In this paper, we propose a novel Stochastic Window Transformer (SWFormer) framework to resolve these issues. First, the effective spatial and spectral feature projection networks are built independently based on hybrid-modal heterogeneous data composition using parallel feature extraction, which is conducive to excavating the perceptual features more representative along different dimensions. Furthermore, to construct local-global nonlinear feature maps more flexibly, we implement multi-scale strip convolution coupled with a transformer strategy. Moreover, in an innovative random window transformer structure, features are randomly masked to achieve sparse window pruning, alleviating the problem of information density redundancy, and reducing the parameters required for intensive attention. Finally, we designed a plug-and-play feature aggregation module that adapts domain offset between modal features adaptively to minimize semantic gaps between them and enhance the representational ability of the fusion feature. Three fiducial datasets demonstrate the effectiveness of the SWFormer in determining classification results.

Relative position matrix and multi-scale feature fusion for writer-independent online signature verification

Online signature verification (OSV) is widely used in finance, law and other fields, and is one of the important research projects on biological characteristics. However, its data set has a small scale and has high requirements for generalization of certification models. Therefore, how to overcome these problems is of great value to improve the practicality and security of online handwriting signature technology. We propose a writer-independent online handwritten signature verification method, which adopts the relative position matrix method to convert the traditional temporal features into images for processing. This method enriched the features of the signatures, serving the purpose of data augmentation. Then two-dimensional multi-scale feature fusion based Siamese neural network (2D-MFFnet) is built for representing and learning the importance of each channel adaptively combined with the attention mechanism. Finally, a temporal convolutional network is designed to construct the classifier. The results illustrate that compared with traditional time series models, the algorithm has reduced the equal error rate by at least 2.52 % on the open datasets MCYT-100 and SVC2004 task2.

CoRTEx: contrastive learning for representing terms via explanations with applications on constructing biomedical knowledge graphs

OBJECTIVES

Biomedical Knowledge Graphs play a pivotal role in various biomedical research domains. Concurrently, term clustering emerges as a crucial step in constructing these knowledge graphs, aiming to identify synonymous terms. Due to a lack of knowledge, previous contrastive learning models trained with Unified Medical Language System (UMLS) synonyms struggle at clustering difficult terms and do not generalize well beyond UMLS terms. In this work, we leverage the world knowledge from large language models (LLMs) and propose Contrastive Learning for Representing Terms via Explanations (CoRTEx) to enhance term representation and significantly improves term clustering.

MATERIALS AND METHODS

The model training involves generating explanations for a cleaned subset of UMLS terms using ChatGPT. We employ contrastive learning, considering term and explanation embeddings simultaneously, and progressively introduce hard negative samples. Additionally, a ChatGPT-assisted BIRCH algorithm is designed for efficient clustering of a new ontology.

RESULTS

We established a clustering test set and a hard negative test set, where our model consistently achieves the highest F1 score. With CoRTEx embeddings and the modified BIRCH algorithm, we grouped 35 580 932 terms from the Biomedical Informatics Ontology System (BIOS) into 22 104 559 clusters with O(N) queries to ChatGPT. Case studies highlight the model's efficacy in handling challenging samples, aided by information from explanations.

CONCLUSION

By aligning terms to their explanations, CoRTEx demonstrates superior accuracy over benchmark models and robustness beyond its training set, and it is suitable for clustering terms for large-scale biomedical ontologies.

Few-shot bronze vessel classification via siamese fourier networks

Exploring ancient Chinese artifacts is crucial for analyzing East Asian technological development, with bronze vessel being the critical element. Bronze vessels, typically featuring intricate carvings, hold historical significance and provide valuable insights into past civilizations. However, identifying bronze patterns can be challenging for human vision, and most RGB-domain methods fail to capture periodic designs. Addressing these issues, we propose the Siamese Fourier Networks (SFN), a parallel network model designed for few-shot regular pattern classification. The Siamese network can differentiate between intricate shapes, while Fourier features enable the extraction of regular textures. To optimize parallel networks, we combine the BCE loss and focal contrastive loss, balancing positive and negative samples. Moreover, we introduce the Bronze Vessel Dataset, featuring 527 samples with diverse shapes and unbalanced distributions. Extensive experiments with advanced few-shot methods demonstrate the superiority of SFN and focal mechanism, significantly improving accuracy.

Learning locality-sensitive bucketing functions

MOTIVATION

Many tasks in sequence analysis ask to identify biologically related sequences in a large set. The edit distance, being a sensible model for both evolution and sequencing error, is widely used in these tasks as a measure. The resulting computational problem-to recognize all pairs of sequences within a small edit distance-turns out to be exceedingly difficult, since the edit distance is known to be notoriously expensive to compute and that all-versus-all comparison is simply not acceptable with millions or billions of sequences. Among many attempts, we recently proposed the locality-sensitive bucketing (LSB) functions to meet this challenge. Formally, a (d1,d2)-LSB function sends sequences into multiple buckets with the guarantee that pairs of sequences of edit distance at most d1 can be found within a same bucket while those of edit distance at least d2 do not share any. LSB functions generalize the locality-sensitive hashing (LSH) functions and admit favorable properties, with a notable highlight being that optimal LSB functions for certain (d1,d2) exist. LSB functions hold the potential of solving above problems optimally, but the existence of LSB functions for more general (d1,d2) remains unclear, let alone constructing them for practical use.

RESULTS

In this work, we aim to utilize machine learning techniques to train LSB functions. With the development of a novel loss function and insights in the neural network structures that can potentially extend beyond this specific task, we obtained LSB functions that exhibit nearly perfect accuracy for certain (d1,d2), matching our theoretical results, and high accuracy for many others. Comparing to the state-of-the-art LSH method Order Min Hash, the trained LSB functions achieve a 2- to 5-fold improvement on the sensitivity of recognizing similar sequences. An experiment on analyzing erroneous cell barcode data is also included to demonstrate the application of the trained LSB functions.

AVAILABILITY AND IMPLEMENTATION

The code for the training process and the structure of trained models are freely available at https://github.com/Shao-Group/lsb-learn.

Impact of Color Space and Color Resolution on Vehicle Recognition Models

In this study, we analyze both linear and nonlinear color mappings by training on versions of a curated dataset collected in a controlled campus environment. We experiment with color space and color resolution to assess model performance in vehicle recognition tasks. Color encodings can be designed in principle to highlight certain vehicle characteristics or compensate for lighting differences when assessing potential matches to previously encountered objects. The dataset used in this work includes imagery gathered under diverse environmental conditions, including daytime and nighttime lighting. Experimental results inform expectations for possible improvements with automatic color space selection through feature learning. Moreover, we find there is only a gradual decrease in model performance with degraded color resolution, which suggests the need for simplified data collection and processing. By focusing on the most critical features, we could see improved model generalization and robustness, as the model becomes less prone to overfitting to noise or irrelevant details in the data. Such a reduction in resolution will lower computational complexity, leading to quicker training and inference times.

Modeling and anti-sway control method for vertical lift-up/lay-down process of slender-beam payload

The existing crane control methods mainly aim at suppressing the oscillation of the payloads during the transportation process. However, during the vertical lift-up/lay-down process of the slender-beam payload (SBP), such as the installation of the wind turbine towers, rocket assembly, and pipeline laying, one end of the SBP is lifted up and the other end is in contact with the ground. In this situation, the external disturbance may cause the SBP to sway around the contact point. In this paper, a nonlinear three-dimensional dynamic model for the vertical lift-up/lay-down process of the SBP is established. On this basis, a control method for suppressing oscillation during the lift-up/lay-down process of the SBP is proposed by adopting the non-singular terminal sliding mode control (NTSMC). The stability of the NTSMC is proved based on the Lyapunov method. Simulations and experiments are carried out to verify the effectiveness of the proposed method.

Image-based molecular representation learning for drug development: a survey

Artificial intelligence (AI) powered drug development has received remarkable attention in recent years. It addresses the limitations of traditional experimental methods that are costly and time-consuming. While there have been many surveys attempting to summarize related research, they only focus on general AI or specific aspects such as natural language processing and graph neural network. Considering the rapid advance on computer vision, using the molecular image to enable AI appears to be a more intuitive and effective approach since each chemical substance has a unique visual representation. In this paper, we provide the first survey on image-based molecular representation for drug development. The survey proposes a taxonomy based on the learning paradigms in computer vision and reviews a large number of corresponding papers, highlighting the contributions of molecular visual representation in drug development. Besides, we discuss the applications, limitations and future directions in the field. We hope this survey could offer valuable insight into the use of image-based molecular representation learning in the context of drug development.

Lateral Flow Assay Biotesting by Utilizing Plasmonic Nanoparticles Made of Inexpensive Metals─Replacing Colloidal Gold

Nanoparticles (NPs) can be conjugated with diverse biomolecules and employed in biosensing to detect target analytes in biological samples. This proven concept was primarily used during the COVID-19 pandemic with gold-NP-based lateral flow assays (LFAs). Considering the gold price and its worldwide depletion, here we show that novel plasmonic NPs based on inexpensive metals, titanium nitride (TiN) and copper covered with a gold shell (Cu@Au), perform comparable to or even better than gold nanoparticles. After conjugation, these novel nanoparticles provided high figures of merit for LFA testing, such as high signals and specificity and robust naked-eye signal recognition. Since the main cost of Au NPs in commercial testing kits is the colloidal synthesis, our development with the Cu@Au and the laser-ablation-fabricated TiN NPs is exciting, offering potentially inexpensive plasmonic nanomaterials for various bioapplications. Moreover, our machine learning study showed that biodetection with TiN is more accurate than that with Au.

Lateral Flow Assays Biotesting by Utilizing Plasmonic Nanoparticles Made of Inexpensive Metals - Replacing Colloidal Gold

Nanoparticles (NPs) can be conjugated with diverse biomolecules and employed in biosensing to detect target analytes in biological samples. This proven concept was primarily used during the COVID-19 pandemic with gold NPs-based lateral flow assays (LFAs). Considering the gold price and its worldwide depletion, here we show that novel plasmonic nanoparticles (NPs) based on inexpensive metals, titanium nitride (TiN) and copper covered with a gold shell (Cu@Au), perform comparable or even better than gold nanoparticles. After conjugation, these novel nanoparticles provided high figures of merit for LFA testing, such as high signals and specificity and robust naked-eye signal recognition. To the best of our knowledge, our study represents the 1st application of laser-ablation-fabricated nanoparticles (TiN) in the LFA and dot-blot biotesting. Since the main cost of the Au NPs in commercial testing kits is in the colloidal synthesis, our development with TiN is very exciting, offering potentially very inexpensive plasmonic nanomaterials for various bio-testing applications. Moreover, our machine learning study showed that the bio-detection with TiN is more accurate than that with Au.

A Siamese ResNeXt network for predicting carotid intimal thickness of patients with T2DM from fundus images

Objective

To develop and validate an artificial intelligence diagnostic model based on fundus images for predicting Carotid Intima-Media Thickness (CIMT) in individuals with Type 2 Diabetes Mellitus (T2DM).

Methods

In total, 1236 patients with T2DM who had both retinal fundus images and CIMT ultrasound records within a single hospital stay were enrolled. Data were divided into normal and thickened groups and sent to eight deep learning models: convolutional neural networks of the eight models were all based on ResNet or ResNeXt. Their encoder and decoder modes are different, including the standard mode, the Parallel learning mode, and the Siamese mode. Except for the six unimodal networks, two multimodal networks based on ResNeXt under the Parallel learning mode or the Siamese mode were embedded with ages. Performance of eight models were compared via the confusion matrix, precision, recall, specificity, F1 value, and ROC curve, and recall was regarded as the main indicator. Besides, Grad-CAM was used to visualize the decisions made by Siamese ResNeXt network, which is the best performance.

Results

Performance of various models demonstrated the following points: 1) the RexNeXt showed a notable improvement over the ResNet; 2) the structural Siamese networks, which extracted features parallelly and independently, exhibited slight performance enhancements compared to the traditional networks. Notably, the Siamese networks resulted in significant improvements; 3) the performance of classification declined if the age factor was embedded in the network. Taken together, the Siamese ResNeXt unimodal model performed best for its superior efficacy and robustness. This model achieved a recall rate of 88.0% and an AUC value of 90.88% in the validation subset. Additionally, heatmaps calculated by the Grad-CAM algorithm presented concentrated and orderly mappings around the optic disc vascular area in normal CIMT groups and dispersed, irregular patterns in thickened CIMT groups.

Conclusion

We provided a Siamese ResNeXt neural network for predicting the carotid intimal thickness of patients with T2DM from fundus images and confirmed the correlation between fundus microvascular lesions and CIMT.

DeepSLICEM: Clustering CryoEM particles using deep image and similarity graph representations

Finding the 3D structure of proteins and their complexes has several applications, such as developing vaccines that target viral proteins effectively. Methods such as cryogenic electron microscopy (cryo-EM) have improved in their ability to capture high-resolution images, and when applied to a purified sample containing copies of a macromolecule, they can be used to produce a high-quality snapshot of different 2D orientations of the macromolecule, which can be combined to reconstruct its 3D structure. Instead of purifying a sample so that it contains only one macromolecule, a process that can be difficult, time-consuming, and expensive, a cell sample containing multiple particles can be photographed directly and separated into its constituent particles using computational methods. Previous work, SLICEM, has separated 2D projection images of different particles into their respective groups using 2 methods, clustering a graph with edges weighted by pairwise similarities of common lines of the 2D projections. In this work, we develop DeepSLICEM, a pipeline that clusters rich representations of 2D projections, obtained by combining graphical features from a similarity graph based on common lines, with additional image features extracted from a convolutional neural network. DeepSLICEM explores 6 pretrained convolutional neural networks and one supervised Siamese CNN for image representation, 10 pretrained deep graph neural networks for similarity graph node representations, and 4 methods for clustering, along with 8 methods for directly clustering the similarity graph. On 6 synthetic and experimental datasets, the DeepSLICEM pipeline finds 92 method combinations achieving better clustering accuracy than previous methods from SLICEM. Thus, in this paper, we demonstrate that deep neural networks have great potential for accurately separating mixtures of 2D projections of different macromolecules computationally.

Broadening the capture of natural products mentioned in FAERS using fuzzy string-matching and a Siamese neural network

Increased sales of natural products (NPs) in the US and growing safety concerns highlight the need for NP pharmacovigilance. A challenge for NP pharmacovigilance is ambiguity when referring to NPs in spontaneous reporting systems. We used a combination of fuzzy string-matching and a neural network to reduce this ambiguity. Our aim is to increase the capture of reports involving NPs in the US Food and Drug Administration Adverse Event Reporting System (FAERS). For this, we utilized Gestalt pattern-matching (GPM) and Siamese neural network (SM) to identify potential mentions of NPs of interest in 389,386 FAERS reports with unmapped drug names. A team of health professionals refined the candidates identified in the previous step through manual review and annotation. After candidate adjudication, GPM identified 595 unique NP names and SM 504. There was little overlap between candidates identified by each (Non-overlapping: GPM 347, SM 248). We identified a total of 686 novel NP names from FAERS reports. Including these names in the FAERS collection yielded 3,486 additional reports mentioning NPs.

Siamese model for collateral score prediction from computed tomography angiography images in acute ischemic stroke

Introduction

Imaging biomarkers, such as the collateral score as determined from Computed Tomography Angiography (CTA) images, play a role in treatment decision making for acute stroke patients. In this manuscript, we present an end-to-end learning approach for automatic determination of a collateral score from a CTA image. Our aim was to investigate whether such end-to-end learning approaches can be used for this classification task, and whether the resulting classification can be used in existing outcome prediction models.

Methods

The method consists of a preprocessing step, where the CTA image is aligned to an atlas and divided in the two hemispheres: the affected side and the healthy side. Subsequently, a VoxResNet based convolutional neural network is used to extract features at various resolutions from the input images. This is done by using a Siamese model, such that the classification is driven by the comparison between the affected and healthy using a unique set of features for both hemispheres. After masking the resulting features for both sides with the vascular region and global average pooling (per hemisphere) and concatenation of the resulting features, a fully connected layer is used to determine the categorized collateral score.

Experiments

Several experiments have been performed to optimize the model hyperparameters and training procedure, and to validate the final model performance. The hyperparameter optimization and subsequent model training was done using CTA images from the MR CLEAN Registry, a Dutch multi-center multi-vendor registry of acute stroke patients that underwent endovascular treatment. A separate set of images, from the MR CLEAN Trial, served as an external validation set, where collateral scoring was assessed and compared with both human observers and a recent more traditional model. In addition, the automated collateral scores have been used in an existing functional outcome prediction model that uses both imaging and non-imaging clinical parameters.

Conclusion

The results show that end-to-end learning of collateral scoring in CTA images is feasible, and does perform similar to more traditional methods, and the performance also is within the inter-observer variation. Furthermore, the results demonstrate that the end-to-end classification results also can be used in an existing functional outcome prediction model.

Disentangling Accelerated Cognitive Decline from the Normal Aging Process and Unraveling Its Genetic Components: A Neuroimaging-Based Deep Learning Approach

Background

The progressive cognitive decline, an integral component of Alzheimer's disease (AD), unfolds in tandem with the natural aging process. Neuroimaging features have demonstrated the capacity to distinguish cognitive decline changes stemming from typical brain aging and AD between different chronological points.

Objective

To disentangle the normal aging effect from the AD-related accelerated cognitive decline and unravel its genetic components using a neuroimaging-based deep learning approach.

Methods

We developed a deep-learning framework based on a dual-loss Siamese ResNet network to extract fine-grained information from the longitudinal structural magnetic resonance imaging (MRI) data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) study. We then conducted genome-wide association studies (GWAS) and post-GWAS analyses to reveal the genetic basis of AD-related accelerated cognitive decline.

Results

We used our model to process data from 1,313 individuals, training it on 414 cognitively normal people and predicting cognitive assessment for all participants. In our analysis of accelerated cognitive decline GWAS, we identified two genome-wide significant loci: APOE locus (chromosome 19 p13.32) and rs144614292 (chromosome 11 p15.1). Variant rs144614292 (G > T) has not been reported in previous AD GWA studies. It is within the intronic region of NELL1, which is expressed in neurons and plays a role in controlling cell growth and differentiation. The cell-type-specific enrichment analysis and functional enrichment of GWAS signals highlighted the microglia and immune-response pathways.

Conclusions

Our deep learning model effectively extracted relevant neuroimaging features and predicted individual cognitive decline. We reported a novel variant (rs144614292) within the NELL1 gene.

Multimodal learning in clinical proteomics: enhancing antimicrobial resistance prediction models with chemical information

MOTIVATION

Large-scale clinical proteomics datasets of infectious pathogens, combined with antimicrobial resistance outcomes, have recently opened the door for machine learning models which aim to improve clinical treatment by predicting resistance early. However, existing prediction frameworks typically train a separate model for each antimicrobial and species in order to predict a pathogen's resistance outcome, resulting in missed opportunities for chemical knowledge transfer and generalizability.

RESULTS

We demonstrate the effectiveness of multimodal learning over proteomic and chemical features by exploring two clinically relevant tasks for our proposed deep learning models: drug recommendation and generalized resistance prediction. By adopting this multi-view representation of the pathogenic samples and leveraging the scale of the available datasets, our models outperformed the previous single-drug and single-species predictive models by statistically significant margins. We extensively validated the multi-drug setting, highlighting the challenges in generalizing beyond the training data distribution, and quantitatively demonstrate how suitable representations of antimicrobial drugs constitute a crucial tool in the development of clinically relevant predictive models.

AVAILABILITY AND IMPLEMENTATION

The code used to produce the results presented in this article is available at https://github.com/BorgwardtLab/MultimodalAMR.

Uncovering developmental time and tempo using deep learning

During animal development, embryos undergo complex morphological changes over time. Differences in developmental tempo between species are emerging as principal drivers of evolutionary novelty, but accurate description of these processes is very challenging. To address this challenge, we present here an automated and unbiased deep learning approach to analyze the similarity between embryos of different timepoints. Calculation of similarities across stages resulted in complex phenotypic fingerprints, which carry characteristic information about developmental time and tempo. Using this approach, we were able to accurately stage embryos, quantitatively determine temperature-dependent developmental tempo, detect naturally occurring and induced changes in the developmental progression of individual embryos, and derive staging atlases for several species de novo in an unsupervised manner. Our approach allows us to quantify developmental time and tempo objectively and provides a standardized way to analyze early embryogenesis.

Microscopic image recognition of diatoms based on deep learning

Diatoms are a crucial component in the study of aquatic ecosystems and ancient environmental records. However, traditional methods for identifying diatoms, such as morphological taxonomy and molecular detection, are costly, are time consuming, and have limitations. To address these issues, we developed an extensive collection of diatom images, consisting of 7983 images from 160 genera and 1042 species, which we expanded to 49,843 through preprocessing, segmentation, and data augmentation. Our study compared the performance of different algorithms, including backbones, batch sizes, dynamic data augmentation, and static data augmentation on experimental results. We determined that the ResNet152 network outperformed other networks, producing the most accurate results with top-1 and top-5 accuracies of 85.97% and 95.26%, respectively, in identifying 1042 diatom species. Additionally, we propose a method that combines model prediction and cosine similarity to enhance the model's performance in low-probability predictions, achieving an 86.07% accuracy rate in diatom identification. Our research contributes significantly to the recognition and classification of diatom images and has potential applications in water quality assessment, ecological monitoring, and detecting changes in aquatic biodiversity.

Classifying marine mammals signal using cubic splines interpolation combining with triple loss variational auto-encoder

In practical applications of passive sonar principles for extracting characteristic frequencies of acoustic signals, scientists typically employ traditional time-frequency domain transformation methods such as Mel-frequency, Short time Fourier transform (STFT), and Wavelet transform (WT). However, these solutions still face limitations in resolution and information loss when transforming data collected over extended periods. In this paper, we present a study using a two-stage approach that combines pre-processing by Cubic-splines interpolation (CSI) with a probability distribution in the hidden space with Siamese triple loss network model for classifying marine mammal (MM) communication signals. The Cubic-splines interpolation technique is tested with the STFT transformation to generate STFT-CSI spectrograms, which enforce stronger relationships between characteristic frequencies, enhancing the connectivity of spectrograms and highlighting frequency-based features. Additionally, stacking spectrograms generated by three consecutive methods, Mel, STFT-CSI, and Wavelet, into a feature spectrogram optimizes the advantages of each method across different frequency bands, resulting in a more effective classification process. The proposed solution using an Siamese Neural Network-Variational Auto Encoder (SNN-VAE) model also overcomes the drawbacks of the Auto-Encoder (AE) structure, including loss of discontinuity and loss of completeness during decoding. The classification accuracy of marine mammal signals using the SNN-VAE model increases by 11% and 20% compared to using the AE model (2013), and by 6% compared to using the Resnet model (2022) on the same actual dataset NOAA from the National Oceanic and Atmospheric Administration - United State of America.

Learning from the unknown: exploring the range of bacterial functionality

Determining the repertoire of a microbe's molecular functions is a central question in microbial biology. Modern techniques achieve this goal by comparing microbial genetic material against reference databases of functionally annotated genes/proteins or known taxonomic markers such as 16S rRNA. Here, we describe a novel approach to exploring bacterial functional repertoires without reference databases. Our Fusion scheme establishes functional relationships between bacteria and assigns organisms to Fusion-taxa that differ from otherwise defined taxonomic clades. Three key findings of our work stand out. First, bacterial functional comparisons outperform marker genes in assigning taxonomic clades. Fusion profiles are also better for this task than other functional annotation schemes. Second, Fusion-taxa are robust to addition of novel organisms and are, arguably, able to capture the environment-driven bacterial diversity. Finally, our alignment-free nucleic acid-based Siamese Neural Network model, created using Fusion functions, enables finding shared functionality of very distant, possibly structurally different, microbial homologs. Our work can thus help annotate functional repertoires of bacterial organisms and further guide our understanding of microbial communities.

DrugRep-HeSiaGraph: when heterogenous siamese neural network meets knowledge graphs for drug repurposing

BACKGROUND

Drug repurposing is an approach that holds promise for identifying new therapeutic uses for existing drugs. Recently, knowledge graphs have emerged as significant tools for addressing the challenges of drug repurposing. However, there are still major issues with constructing and embedding knowledge graphs.

RESULTS

This study proposes a two-step method called DrugRep-HeSiaGraph to address these challenges. The method integrates the drug-disease knowledge graph with the application of a heterogeneous siamese neural network. In the first step, a drug-disease knowledge graph named DDKG-V1 is constructed by defining new relationship types, and then numerical vector representations for the nodes are created using the distributional learning method. In the second step, a heterogeneous siamese neural network called HeSiaNet is applied to enrich the embedding of drugs and diseases by bringing them closer in a new unified latent space. Then, it predicts potential drug candidates for diseases. DrugRep-HeSiaGraph achieves impressive performance metrics, including an AUC-ROC of 91.16%, an AUC-PR of 90.32%, an accuracy of 84.63%, a BS of 0.119, and an MCC of 69.31%.

CONCLUSION

We demonstrate the effectiveness of the proposed method in identifying potential drugs for COVID-19 as a case study. In addition, this study shows the role of dipeptidyl peptidase 4 (DPP-4) as a potential receptor for SARS-CoV-2 and the effectiveness of DPP-4 inhibitors in facing COVID-19. This highlights the practical application of the model in addressing real-world challenges in the field of drug repurposing. The code and data for DrugRep-HeSiaGraph are publicly available at https://github.com/CBRC-lab/DrugRep-HeSiaGraph .

DFU-Helper: An Innovative Framework for Longitudinal Diabetic Foot Ulcer Diseases Evaluation Using Deep Learning

Diabetes affects roughly 537 million people, and is predicted to reach 783 million by 2045. Diabetes Foot Ulcer (DFU) is a major complication associated with diabetes and can lead to lower limb amputation. The rapid evolution of diabetic foot ulcers (DFUs) necessitates immediate intervention to prevent the severe consequences of amputation and related complications. Continuous and meticulous patient monitoring for individuals with diabetic foot ulcers (DFU) is crucial and is currently carried out by medical practitioners on a daily basis. This research article introduces DFU-Helper, a novel framework that employs a Siamese Neural Network (SNN) for accurate and objective assessment of the progression of diabetic foot ulcers (DFUs) over time. DFU-Helper provides healthcare professionals with a comprehensive visual and numerical representation in terms of the similarity distance of the disease, considering five distinct disease conditions: none, infection, ischemia, both (presence of ischemia and infection), and healthy. The SNN achieves the best Macro F1-score of 0.6455 on the test dataset when applying pseudo-labeling with a pseudo-threshold set to 0.9. The SNN is used in the process of creating anchors for each class using feature vectors. When a patient initially consults a healthcare professional, an image is transmitted to the model, which computes the distances from each class anchor point. It generates a comprehensive table with corresponding figures and a visually intuitive radar chart. In subsequent visits, another image is captured and fed into the model alongside the initial image. DFU-Helper then plots both images and presents the distances from the class anchor points. Our proposed system represents a significant advancement in the application of deep learning for the longitudinal assessment of DFU. To the best of our knowledge, no existing tool harnesses deep learning for DFU follow-up in a comparable manner.

Disentangling accelerated cognitive decline from the normal aging process and unraveling its genetic components: A neuroimaging-based deep learning approach

Background

The progressive cognitive decline that is an integral component of AD unfolds in tandem with the natural aging process. Neuroimaging features have demonstrated the capacity to distinguish cognitive decline changes stemming from typical brain aging and Alzheimer's disease between different chronological points.

Methods

We developed a deep-learning framework based on dual-loss Siamese ResNet network to extract fine-grained information from the longitudinal structural magnetic resonance imaging (MRI) data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) study. We then conducted genome-wide association studies (GWAS) and post-GWAS analyses to reveal the genetic basis of AD-related accelerated cognitive decline.

Results

We used our model to process data from 1,313 individuals, training it on 414 cognitively normal people and predicting cognitive assessment for all participants. In our analysis of accelerated cognitive decline GWAS, we identified two genome-wide significant loci: APOE locus (chromosome 19 p13.32) and rs144614292 (chromosome 11 p15.1). Variant rs144614292 (G>T) has not been reported in previous AD GWA studies. It is within the intronic region of NELL1, which is expressed in neuron and plays a role in controlling cell growth and differentiation. In addition, MUC7 and PROL1/OPRPNon chromosome 4 were significant at the gene level. The cell-type-specific enrichment analysis and functional enrichment of GWAS signals highlighted the microglia and immune-response pathways. Furthermore, we found that the cognitive decline slope GWAS was positively correlated with previous AD GWAS.

Conclusion

Our deep learning model was demonstrated effective on extracting relevant neuroimaging features and predicting individual cognitive decline. We reported a novel variant (rs144614292) within the NELL1 gene. Our approach has the potential to disentangle accelerated cognitive decline from the normal aging process and to determine its related genetic factors, leveraging opportunities for early intervention.

ISA: Ingenious Siamese Attention for object detection algorithms towards complex scenes

The interference of complex environments on object detection tasks dramatically limits the application of object detection algorithms. Improving the detection accuracy of the object detection algorithms is able to effectively enhance the stability and reliability of the object detection algorithm-based tasks in complex environments. In order to ameliorate the detection accuracy of object detection algorithms under various complex environment transformations, this work proposes the Siamese Attention YOLO (SAYOLO) object detection algorithm based on ingenious siamese attention structure. The ingenious siamese attention structure includes three aspects: Attention Neck YOLOv4 (ANYOLOv4), siamese neural network structure and special designed network scoring module. In the Complex Mini VOC dataset, the detection accuracy of SAYOLO algorithm is 12.31%, 48.93%, 17.80%, 10.12%, 18.79% and 1.12% higher than Faster-RCNN (Resnet50), SSD (Mobilenetv2), YOLOv3, YOLOv4, YOLOv5-l and YOLOX-x, respectively. Compared with traditional object detection algorithms based on image preprocessing, the detection accuracy of SAYOLO is 4.88%, 11.51%, 1.73%, 23.27%, 18.12%, and 5.76% higher than Image-Adaptive YOLO, MSBDN-DFF + YOLOv4, Dark Channel Prior + YOLOv4, Zero-DCE + YOLOv4, MSBDN-DFF + Zero-DCE + YOLOv4, and Dark Channel Prior + Zero-DCE + YOLOv4, respectively.

Metric networks for enhanced perception of non-local semantic information

Introduction

Metric learning, as a fundamental research direction in the field of computer vision, has played a crucial role in image matching. Traditional metric learning methods aim at constructing two-branch siamese neural networks to address the challenge of image matching, but they often overlook to cross-source and cross-view scenarios.

Methods

In this article, a multi-branch metric learning model is proposed to address these limitations. The main contributions of this work are as follows: Firstly, we design a multi-branch siamese network model that enhances measurement reliability through information compensation among data points. Secondly, we construct a non-local information perception and fusion model, which accurately distinguishes positive and negative samples by fusing information at different scales. Thirdly, we enhance the model by integrating semantic information and establish an information consistency mapping between multiple branches, thereby improving the robustness in cross-source and cross-view scenarios.

Results

Experimental tests which demonstrate the effectiveness of the proposed method are carried out under various conditions, including homologous, heterogeneous, multi-view, and crossview scenarios. Compared to the state-of-the-art comparison algorithms, our proposed algorithm achieves an improvement of ~1, 2, 1, and 1% in terms of similarity measurement Recall@10, respectively, under these four conditions.

Discussion

In addition, our work provides an idea for improving the crossscene application ability of UAV positioning and navigation algorithm.

Discovering Reaction Pathways, Slow Variables, and Committor Probabilities with Machine Learning

A significant challenge faced by atomistic simulations is the difficulty, and often impossibility, to sample the transitions between metastable states of the free-energy landscape associated with slow molecular processes. Importance-sampling schemes represent an appealing option to accelerate the underlying dynamics by smoothing out the relevant free-energy barriers, but require the definition of suitable reaction-coordinate (RC) models expressed in terms of compact low-dimensional sets of collective variables (CVs). While most computational studies of slow molecular processes have traditionally relied on educated guesses based on human intuition to reduce the dimensionality of the problem at hand, a variety of machine-learning (ML) algorithms have recently emerged as powerful alternatives to discover meaningful CVs capable of capturing the dynamics of the slowest degrees of freedom. Considering a simple paradigmatic situation in which the long-time dynamics is dominated by the transition between two known metastable states, we compare two variational data-driven ML methods based on Siamese neural networks aimed at discovering a meaningful RC model─the slowest decorrelating CV of the molecular process, and the committor probability to first reach one of the two metastable states. One method is the state-free reversible variational approach for Markov processes networks (VAMPnets), or SRVs─the other, inspired by the transition path theory framework, is the variational committor-based neural networks, or VCNs. The relationship and the ability of these methodologies to discover the relevant descriptors of the slow molecular process of interest are illustrated with a series of simple model systems. We also show that both strategies are amenable to importance-sampling schemes through an appropriate reweighting algorithm that approximates the kinetic properties of the transition.

Identifying SARS-CoV-2 infected cells with scVDN

Introduction

Single-cell RNA sequencing (scRNA-seq) is a powerful tool for understanding cellular heterogeneity and identifying cell types in virus-related research. However, direct identification of SARS-CoV-2-infected cells at the single-cell level remains challenging, hindering the understanding of viral pathogenesis and the development of effective treatments.

Methods

In this study, we propose a deep learning framework, the single-cell virus detection network (scVDN), to predict the infection status of single cells. The scVDN is trained on scRNA-seq data from multiple nasal swab samples obtained from several contributors with varying cell types. To objectively evaluate scVDN's performance, we establish a model evaluation framework suitable for real experimental data.

Results and Discussion

Our results demonstrate that scVDN outperforms four state-of-the-art machine learning models in identifying SARS-CoV-2-infected cells, even with extremely imbalanced labels in real data. Specifically, scVDN achieves a perfect AUC score of 1 in four cell types. Our findings have important implications for advancing virus research and improving public health by enabling the identification of virus-infected cells at the single-cell level, which is critical for diagnosing and treating viral infections. The scVDN framework can be applied to other single-cell virus-related studies, and we make all source code and datasets publicly available on GitHub at https://github.com/studentiz/scvdn.

Predictive Models for Health Deterioration: Understanding Disease Pathways for Personalized Medicine

Artificial intelligence (AI) and machine learning (ML) methods are currently widely employed in medicine and healthcare. A PubMed search returns more than 100,000 articles on these topics published between 2018 and 2022 alone. Notwithstanding several recent reviews in various subfields of AI and ML in medicine, we have yet to see a comprehensive review around the methods' use in longitudinal analysis and prediction of an individual patient's health status within a personalized disease pathway. This review seeks to fill that gap. After an overview of the AI and ML methods employed in this field and of specific medical applications of models of this type, the review discusses the strengths and limitations of current studies and looks ahead to future strands of research in this field. We aim to enable interested readers to gain a detailed impression of the research currently available and accordingly plan future work around predictive models for deterioration in health status.

Autonomous face mask detection using single shot multibox detector, and ResNet-50 with identity retrieval through face matching using deep siamese neural network

The COVID-19 pandemic poses a global health challenge. The World Health Organization states that face masks are proven to be effective, especially in public areas. Real-time monitoring of face masks is challenging and exhaustive for humans. To reduce human effort and to provide an enforcement mechanism, an autonomous system has been proposed to detect non-masked people and retrieve their identity using computer vision. The proposed method introduces a novel and efficient method that involves fine-tuning the pre-trained ResNet-50 model with a new head layer for classification between masked and non-masked people. The classifier is trained using adaptive momentum optimization algorithm with decaying learning rate and binary cross-entropy loss. Data augmentation and dropout regularization are employed to achieve best convergence. During real-time application of our classifier on videos, a Caffe face detector model based on Single Shot MultiBox Detector is used to extract the face regions of interest from each frame, on which the trained classifier is applied for detecting the non-masked people. The faces of these people are then captured, which is passed on to a deep siamese neural network, based on VGG-Face model for face matching. The captured faces are compared with the reference images from the database, by extracting the features and calculating cosine distance. If the faces match, the details of that person are retrieved from the database and displayed on the web application. The proposed method has secured best results where the trained classifier has achieved 99.74% accuracy, and the identity retrieval model achieved 98.24% accuracy.

SPT: Single Pedestrian Tracking Framework with Re-Identification-Based Learning Using the Siamese Model

Pedestrian tracking is a challenging task in the area of visual object tracking research and it is a vital component of various vision-based applications such as surveillance systems, human-following robots, and autonomous vehicles. In this paper, we proposed a single pedestrian tracking (SPT) framework for identifying each instance of a person across all video frames through a tracking-by-detection paradigm that combines deep learning and metric learning-based approaches. The SPT framework comprises three main modules: detection, re-identification, and tracking. Our contribution is a significant improvement in the results by designing two compact metric learning-based models using Siamese architecture in the pedestrian re-identification module and combining one of the most robust re-identification models for data associated with the pedestrian detector in the tracking module. We carried out several analyses to evaluate the performance of our SPT framework for single pedestrian tracking in the videos. The results of the re-identification module validate that our two proposed re-identification models surpass existing state-of-the-art models with increased accuracies of 79.2% and 83.9% on the large dataset and 92% and 96% on the small dataset. Moreover, the proposed SPT tracker, along with six state-of-the-art (SOTA) tracking models, has been tested on various indoor and outdoor video sequences. A qualitative analysis considering six major environmental factors verifies the effectiveness of our SPT tracker under illumination changes, appearance variations due to pose changes, changes in target position, and partial occlusions. In addition, quantitative analysis based on experimental results also demonstrates that our proposed SPT tracker outperforms the GOTURN, CSRT, KCF, and SiamFC trackers with a success rate of 79.7% while beating the DiamSiamRPN, SiamFC, CSRT, GOTURN, and SiamMask trackers with an average of 18 tracking frames per second.

A gene regulatory network inference model based on pseudo-siamese network

MOTIVATION

Gene regulatory networks (GRNs) arise from the intricate interactions between transcription factors (TFs) and their target genes during the growth and development of organisms. The inference of GRNs can unveil the underlying gene interactions in living systems and facilitate the investigation of the relationship between gene expression patterns and phenotypic traits. Although several machine-learning models have been proposed for inferring GRNs from single-cell RNA sequencing (scRNA-seq) data, some of these models, such as Boolean and tree-based networks, suffer from sensitivity to noise and may encounter difficulties in handling the high noise and dimensionality of actual scRNA-seq data, as well as the sparse nature of gene regulation relationships. Thus, inferring large-scale information from GRNs remains a formidable challenge.

RESULTS

This study proposes a multilevel, multi-structure framework called a pseudo-Siamese GRN (PSGRN) for inferring large-scale GRNs from time-series expression datasets. Based on the pseudo-Siamese network, we applied a gated recurrent unit to capture the time features of each TF and target matrix and learn the spatial features of the matrices after merging by applying the DenseNet framework. Finally, we applied a sigmoid function to evaluate interactions. We constructed two maize sub-datasets, including gene expression levels and GRNs, using existing open-source maize multi-omics data and compared them to other GRN inference methods, including GENIE3, GRNBoost2, nonlinear ordinary differential equations, CNNC, and DGRNS. Our results show that PSGRN outperforms state-of-the-art methods. This study proposed a new framework: a PSGRN that allows GRNs to be inferred from scRNA-seq data, elucidating the temporal and spatial features of TFs and their target genes. The results show the model's robustness and generalization, laying a theoretical foundation for maize genotype-phenotype associations with implications for breeding work.

Deep Learning-based Approach to Predict Pulmonary Function at Chest CT

Background Low-dose chest CT screening is recommended for smokers with the potential for lung function abnormality, but its role in predicting lung function remains unclear. Purpose To develop a deep learning algorithm to predict pulmonary function with low-dose CT images in participants using health screening services. Materials and Methods In this retrospective study, participants underwent health screening with same-day low-dose CT and pulmonary function testing with spirometry at a university affiliated tertiary referral general hospital between January 2015 and December 2018. The data set was split into a development set (model training, validation, and internal test sets) and temporally independent test set according to first visit year. A convolutional neural network was trained to predict the forced expiratory volume in the first second of expiration (FEV₁) and forced vital capacity (FVC) from low-dose CT. The mean absolute error and concordance correlation coefficient (CCC) were used to evaluate agreement between spirometry as the reference standard and deep-learning prediction as the index test. FVC and FEV₁ percent predicted (hereafter, FVC% and FEV₁%) values less than 80% and percent of FVC exhaled in first second (hereafter, FEV₁/FVC) less than 70% were used to classify participants at high risk. Results A total of 16 148 participants were included (mean age, 55 years ± 10 [SD]; 10 981 men) and divided into a development set (n = 13 428) and temporally independent test set (n = 2720). In the temporally independent test set, the mean absolute error and CCC were 0.22 L and 0.94, respectively, for FVC and 0.22 L and 0.91 for FEV₁. For the prediction of the respiratory high-risk group, FVC%, FEV₁%, and FEV₁/FVC had respective accuracies of 89.6% (2436 of 2720 participants; 95% CI: 88.4, 90.7), 85.9% (2337 of 2720 participants; 95% CI: 84.6, 87.2), and 90.2% (2453 of 2720 participants; 95% CI: 89.1, 91.3) in the same testing data set. The sensitivities were 61.6% (242 of 393 participants; 95% CI: 59.7, 63.4), 46.9% (226 of 482 participants; 95% CI: 45.0, 48.8), and 36.1% (91 of 252 participants; 95% CI: 34.3, 37.9), respectively. Conclusion A deep learning model applied to volumetric chest CT predicted pulmonary function with relatively good performance. © RSNA, 2023 Supplemental material is available for this article.

Chasing collective variables using temporal data-driven strategies

The convergence of free-energy calculations based on importance sampling depends heavily on the choice of collective variables (CVs), which in principle, should include the slow degrees of freedom of the biological processes to be investigated. Autoencoders (AEs), as emerging data-driven dimension reduction tools, have been utilised for discovering CVs. AEs, however, are often treated as black boxes, and what AEs actually encode during training, and whether the latent variables from encoders are suitable as CVs for further free-energy calculations remains unknown. In this contribution, we review AEs and their time-series-based variants, including time-lagged AEs (TAEs) and modified TAEs, as well as the closely related model variational approach for Markov processes networks (VAMPnets). We then show through numerical examples that AEs learn the high-variance modes instead of the slow modes. In stark contrast, time series-based models are able to capture the slow modes. Moreover, both modified TAEs with extensions from slow feature analysis and the state-free reversible VAMPnets (SRVs) can yield orthogonal multidimensional CVs. As an illustration, we employ SRVs to discover the CVs of the isomerizations of N-acetyl-N'-methylalanylamide and trialanine by iterative learning with trajectories from biased simulations. Last, through numerical experiments with anisotropic diffusion, we investigate the potential relationship of time-series-based models and committor probabilities.

Single-cell multi-omics integration for unpaired data by a siamese network with graph-based contrastive loss

BACKGROUND

Single-cell omics technology is rapidly developing to measure the epigenome, genome, and transcriptome across a range of cell types. However, it is still challenging to integrate omics data from different modalities. Here, we propose a variation of the Siamese neural network framework called MinNet, which is trained to integrate multi-omics data on the single-cell resolution by using graph-based contrastive loss.

RESULTS

By training the model and testing it on several benchmark datasets, we showed its accuracy and generalizability in integrating scRNA-seq with scATAC-seq, and scRNA-seq with epitope data. Further evaluation demonstrated our model's unique ability to remove the batch effect, a common problem in actual practice. To show how the integration impacts downstream analysis, we established model-based smoothing and cis-regulatory element-inferring method and validated it with external pcHi-C evidence. Finally, we applied the framework to a COVID-19 dataset to bolster the original work with integration-based analysis, showing its necessity in single-cell multi-omics research.

CONCLUSIONS

MinNet is a novel deep-learning framework for single-cell multi-omics sequencing data integration. It ranked top among other methods in benchmarking and is especially suitable for integrating datasets with batch and biological variances. With the single-cell resolution integration results, analysis of the interplay between genome and transcriptome can be done to help researchers understand their data and question.

Siam Deep Feature KCF Method and Experimental Study for Pedestrian Tracking

The tracking of a particular pedestrian is an important issue in computer vision to guarantee societal safety. Due to the limited computing performances of unmanned aerial vehicle (UAV) systems, the Correlation Filter (CF) algorithm has been widely used to perform the task of tracking. However, it has a fixed template size and cannot effectively solve the occlusion problem. Thus, a tracking-by-detection framework was designed in the current research. A lightweight YOLOv3-based (You Only Look Once version 3) mode which had Efficient Channel Attention (ECA) was integrated into the CF algorithm to provide deep features. In addition, a lightweight Siamese CNN with Cross Stage Partial (CSP) provided the representations of features learned from massive face images, allowing the target similarity in data association to be guaranteed. As a result, a Deep Feature Kernelized Correlation Filters method coupled with Siamese-CSP(Siam-DFKCF) was established to increase the tracking robustness. From the experimental results, it can be concluded that the anti-occlusion and re-tracking performance of the proposed method was increased. The tracking accuracy Distance Precision (DP) and Overlap Precision (OP) had been increased to 0.934 and 0.909 respectively in our test data.

Robust Wheel Detection for Vehicle Re-Identification

Vehicle re-identification is a demanding and challenging task in automated surveillance systems. The goal of vehicle re-identification is to associate images of the same vehicle to identify re-occurrences of the same vehicle. Robust re-identification of individual vehicles requires reliable and discriminative features extracted from specific parts of the vehicle. In this work, we construct an efficient and robust wheel detector that precisely locates and selects vehicular wheels from vehicle images. The associated hubcap geometry can hence be utilized to extract fundamental signatures from vehicle images and exploit them for vehicle re-identification. Wheels pattern information can yield additional information about vehicles in questions. To that end, we utilized a vehicle imagery dataset that has thousands of side-view vehicle collected under different illumination conditions and elevation angles. The collected dataset was used for training and testing the wheel detector. Experiments show that our approach could detect vehicular wheels accurately for 99.41% of the vehicles in the dataset.