Object-adaptive LSTM network for real-time visual tracking with adversarial data augmentation

Firefly algorithm-based LSTM model for Guzheng tunes switching with big data analysis

Guzheng tune progression involves intricately harmonizing melodic motif transitions. Effectively navigating this vast creative possibility space to expose musically consistent elaborations presents challenges. We develop a specialized large long short-term memory (LSTM) model for generating musically consistent Guzheng tune transitions. First, we propose novel firefly algorithm (FA) enhancements, e.g., adaptive diversity preservation and adaptive swim parameters, to boost exploration effectiveness for navigating the vast creative combinatorics when generating Guzheng tune transitions. Then, we develop a specialized stacked LSTM architecture incorporating residual connections and conditioned embedding vectors that can leverage long-range temporal dependencies in Guzheng music patterns, including unsupervised learning of concise Guzheng-specific melody embedding vectors via a variational autoencoder, encapsulating unique harmonic signatures from performance descriptors to provide style guidance. Finally, we use LSTM networks to develop adversarial generative large models that enable realistic synthesis and evaluation of Guzheng tunes switching. We gather an extensive 10+ hour corpus of solo Guzheng recordings spanning 230 musical pieces, 130 distinguished performing artists, and 600+ audio tracks. Simultaneously, we conduct thorough Guzheng data analysis. Comparative assessments against strong baselines over systematic musical metrics and professional listeners validate significant generation fidelity improvements. Our model achieves a 63 % reduction in reconstruction error compared to the standard FA optimization after 1000 iterations. It also outperforms baselines in capturing characteristic motifs, maintaining modality coherence with under 2 % dissonant pitch errors, and retaining desired rhythmic cadences. User studies further confirm the superior naturalness, novelty, and stylistic faithfulness of the generated tune transitions, with ratings close to real data.

Research Ideas Discovery via Hierarchical Negative Correlation

A new research idea may be inspired by the connections of keywords. Link prediction discovers potential nonexisting links in an existing graph and has been applied in many applications. This article explores a method of discovering new research ideas based on link prediction, which predicts the possible connections of different keywords by analyzing the topological structure of the keyword graph. The patterns of links between keywords may be diversified due to different domains and different habits of authors. Therefore, it is often difficult for a single learner to extract diverse patterns of different research domains. To address this issue, groups of learners are organized with negative correlation to encourage the diversity of sublearners. Moreover, a hierarchical negative correlation mechanism is proposed to extract subgraph features in different order subgraphs, which improves the diversity by explicitly supervising the negative correlation on each layer of sublearners. Experiments are conducted to illustrate the effectiveness of the proposed model to discover new research ideas. Under the premise of ensuring the performance of the model, the proposed method consumes less time and computational cost compared with other ensemble methods.

Estimation of Joint Torque by EMG-Driven Neuromusculoskeletal Models and LSTM Networks

Accurately predicting joint torque using wearable sensors is crucial for designing assist-as-needed exoskeleton controllers to assist muscle-generated torque and ensure successful task performance. In this paper, we estimated ankle dorsiflexion/plantarflexion, knee flexion/extension, hip flexion/extension, and hip abduction/adduction torques from electromyography (EMG) and kinematics during daily activities using neuromusculoskeletal (NMS) models and long short-term memory (LSTM) networks. The joint torque ground truth for model calibrating and training was obtained through inverse dynamics of captured motion data. A cluster approach that grouped movements based on characteristic similarity was implemented, and its ability to improve the estimation accuracy of both NMS and LSTM models was evaluated. We compared torque estimation accuracy of NMS and LSTM models in three cases: Pooled, Individual, and Clustered models. Pooled models used data from all 10 movements to calibrate or train one model, Individual models used data from each individual movement, and Clustered models used data from each cluster. Individual, Clustered and Pooled LSTM models all had relatively high joint torque estimation accuracy. Individual and Clustered NMS models had similarly good estimation performance whereas the Pooled model may be too generic to satisfy all movement patterns. While the cluster approach improved the estimation accuracy in NMS models in some movements, it made relatively little difference in the LSTM neural networks, which already had high estimation accuracy. Our study provides practical implications for designing assist-as-needed exoskeleton controllers by offering guidelines for selecting the appropriate model for different scenarios, and has potential to enhance the functionality of wearable exoskeletons and improve rehabilitation and assistance for individuals with motor disorders.

Multiobject Tracking of Wildlife in Videos Using Few-Shot Learning

Simple Summary

Video recordings enable scientists to estimate species’ presence, richness, abundance, demography, and activity. The increasing popularity of camera traps has led to a growing interest in developing approaches to more efficiently process images. Advanced artificial intelligence systems can automatically find and identify the species captured in the wild, but they are hampered by dependence on large samples. However, many species rarely occur, such as endangered species, and only a few shot samples are available. Building on recent advances in deep learning and few-shot learning technologies, we developed a multiobject-tracking approach based on a tracking-by-detection paradigm for wildlife to improve multiobject-tracking performance. We hope that it will be beneficial to ecology and wildlife biology by speeding up the process of multiobject tracking in the wild.

Abstract

Camera trapping and video recording are now ubiquitous in the study of animal ecology. These technologies hold great potential for wildlife tracking, but are limited by current learning approaches, and are hampered by dependence on large samples. Most species of wildlife are rarely captured by camera traps, and thus only a few shot samples are available for processing and subsequent identification. These drawbacks can be overcome in multiobject tracking by combining wildlife detection and tracking with few-shot learning. This work proposes a multiobject-tracking approach based on a tracking-by-detection paradigm for wildlife to improve detection and tracking performance. We used few-shot object detection to localize objects using a camera trap and direct video recordings that could augment the synthetically generated parts of separate images with spatial constraints. In addition, we introduced a trajectory reconstruction module for better association. It could alleviate a few-shot object detector’s missed and false detections; in addition, it could optimize the target identification between consecutive frames. Our approach produced a fully automated pipeline for detecting and tracking wildlife from video records. The experimental results aligned with theoretical anticipation according to various evaluation metrics, and revealed the future potential of camera traps to address wildlife detection and tracking in behavior and conservation.

A Review of Tracking and Trajectory Prediction Methods for Autonomous Driving

This paper provides a literature review of some of the most important concepts, techniques, and methodologies used within autonomous car systems. Specifically, we focus on two aspects extensively explored in the related literature: tracking, i.e., identifying pedestrians, cars or obstacles from images, observations or sensor data, and prediction, i.e., anticipating the future trajectories and motion of other vehicles in order to facilitate navigating through various traffic conditions. Approaches based on deep neural networks and others, especially stochastic techniques, are reported.