Active Learning for Deep Visual Tracking

Cross-Modal Object Tracking via Modality-Aware Fusion Network and a Large-Scale Dataset

Visual object tracking often faces challenges such as invalid targets and decreased performance in low-light conditions when relying solely on RGB image sequences. While incorporating additional modalities like depth and infrared data has proven effective, existing multimodal imaging platforms are complex and lack real-world applicability. In contrast, near-infrared (NIR) imaging, commonly used in surveillance cameras, can switch between RGB and NIR based on light intensity. However, tracking objects across these heterogeneous modalities poses significant challenges, particularly due to the absence of modality switch signals during tracking. To address these challenges, we propose an adaptive cross-modal object tracking algorithm called modality-aware fusion network (MAFNet). MAFNet efficiently integrates information from both RGB and NIR modalities using an adaptive weighting mechanism, effectively bridging the appearance gap and enabling a modality-aware target representation. It consists of two key components: an adaptive weighting module and a modality-specific representation module. The adaptive weighting module predicts fusion weights to dynamically adjust the contribution of each modality, while the modality-specific representation module captures discriminative features specific to RGB and NIR modalities. MAFNet offers great flexibility as it can effortlessly integrate into diverse tracking frameworks. With its simplicity, effectiveness, and efficiency, MAFNet outperforms state-of-the-art methods in cross-modal object tracking. To validate the effectiveness of our algorithm and overcome the scarcity of data in this field, we introduce CMOTB, a comprehensive and extensive benchmark dataset for cross-modal object tracking. CMOTB consists of 61 categories and 1000 video sequences, comprising a total of over 799K frames. We believe that our proposed method and dataset offer a strong foundation for advancing cross-modal object-tracking research. The dataset, toolkit, experimental data, and source code will be publicly available at: https://github.com/mmic-lcl/ Datasets-and-benchmark-code.

EnhanceCenter for improving point based tracking and rich feature representation

In this study, we propose EnhanceCenter, a multiple-object tracking model that demonstrates enhanced tracking efficiency and stability while reducing dependencies on computationally intensive detectors. EnhanceCenter, based on the CenterTrack method, introduces three key improvements. First, a channel-spatial-spatial feature fusion module effectively utilizes object appearance information, enhancing tracking in complex scenes. Second, the backbone network weights are optimized for multiple-object tracking tasks, enabling more effective feature extraction. Lastly, an improved association method increases long-term tracking stability, maintaining consistency during occlusions or detection failures. Experiments on various MOT benchmarks demonstrated the performance of EnhanceCenter against models using high-performance detectors. On the MOT17 test set, EnhanceCenter outperformed CenterTrack with a 1.6% improvement in IDF1 and achieved a HOTA of 55.1%, surpassing leading center-point-based tracking studies, such as TransTrack and TransCenter. The MOT20 dataset showed a significant 13% improvement in IDF1 compared to CenterTrack. This research underscores the potential of lightweight detectors in achieving state-of-the-art multiple-object tracking performance, paving the way for more efficient tracking solutions in complex environments.

Cooperative Active Learning-Based Dual Control for Exploration and Exploitation in Autonomous Search

In this article, a multi-estimator based computationally efficient algorithm is developed for autonomous search in an unknown environment with an unknown source. Different from the existing approaches that require massive computational power to support nonlinear Bayesian estimation and complex decision-making process, an efficient cooperative active-learning-based dual control for exploration and exploitation (COAL-DCEE) is developed for source estimation and path planning. Multiple cooperative estimators are deployed for environment learning process, which is helpful to improving the search performance and robustness against noisy measurements. The number of estimators used in COAL-DCEE is much smaller than that of the particles required for Bayesian estimation in information-theoretic approaches. Consequently, the computational load is significantly reduced. As an important feature of this study, the convergence and performance of COAL-DCEE are established in relation to the characteristics of sensor noises and turbulence disturbances. Numerical and experimental studies have been carried out to verify the effectiveness of the proposed framework. Compared with the existing approaches, COAL-DCEE not only provides convergence guarantee but also yields comparable search performance using much less computational power.

Self-supervised context-aware correlation filter for robust landmark tracking in liver ultrasound sequences

OBJECTIVES

Respiratory motion-induced displacement of internal organs poses a significant challenge in image-guided radiation therapy, particularly affecting liver landmark tracking accuracy.

METHODS

Addressing this concern, we propose a self-supervised method for robust landmark tracking in long liver ultrasound sequences. Our approach leverages a Siamese-based context-aware correlation filter network, trained by using the consistency loss between forward tracking and back verification. By effectively utilizing both labeled and unlabeled liver ultrasound images, our model, Siam-CCF , mitigates the impact of speckle noise and artifacts on ultrasonic image tracking by a context-aware correlation filter. Additionally, a fusion strategy for template patch feature helps the tracker to obtain rich appearance information around the point-landmark.

RESULTS

Siam-CCF achieves a mean tracking error of 0.79 ± 0.83 mm at a frame rate of 118.6 fps, exhibiting a superior speed-accuracy trade-off on the public MICCAI 2015 Challenge on Liver Ultrasound Tracking (CLUST2015) 2D dataset. This performance won the 5th place on the CLUST2015 2D point-landmark tracking task.

CONCLUSIONS

Extensive experiments validate the effectiveness of our proposed approach, establishing it as one of the top-performing techniques on the CLUST2015 online leaderboard at the time of this submission.

STS-TransUNet: Semi-supervised Tooth Segmentation Transformer U-Net for dental panoramic image

In this paper, we introduce a novel deep learning method for dental panoramic image segmentation, which is crucial in oral medicine and orthodontics for accurate diagnosis and treatment planning. Traditional methods often fail to effectively combine global and local context, and struggle with unlabeled data, limiting performance in varied clinical settings. We address these issues with an advanced TransUNet architecture, enhancing feature retention and utilization by connecting the input and output layers directly. Our architecture further employs spatial and channel attention mechanisms in the decoder segments for targeted region focus, and deep supervision techniques to overcome the vanishing gradient problem for more efficient training. Additionally, our network includes a self-learning algorithm using unlabeled data, boosting generalization capabilities. Named the Semi-supervised Tooth Segmentation Transformer U-Net (STS-TransUNet), our method demonstrated superior performance on the MICCAI STS-2D dataset, proving its effectiveness and robustness in tooth segmentation tasks.