Real-World ISAR Object Recognition Using Deep Multimodal Relation Learning

REMI: Few-Shot ISAR Target Classification Via Robust Embedding and Manifold Inference

Unknown image deformation and few-shot issues have posed significant challenges to inverse synthetic aperture radar (ISAR) target classification. To achieve robust feature representation and precise correlation modeling, this article proposes a novel two-stage few-shot ISAR classification network, dubbed as robust embedding and manifold inference (REMI). In the robust embedding stage, a multihead spatial transformation network (MH-STN) is designed to adjust unknown image deformations from multiple perspectives. Then, the grouped embedding network (GEN) integrates and compresses diverse information by grouped feature extraction, intermediate feature fusion, and global feature embedding. In the manifold inference stage, a masked Gaussian graph attention network (MG-GAT) is devised to capture the irregular manifold of samples in the embedding space. In particular, the node features are described by Gaussian distributions, with interactions guided by the masked attention mechanism. Experimental results on two ISAR datasets demonstrate that REMI significantly improves the performance of few-shot classification and exhibits robustness in various scenarios.

Multiattention Mechanism 3D Object Detection Algorithm Based on RGB and LiDAR Fusion for Intelligent Driving

This paper proposes a multimodal fusion 3D target detection algorithm based on the attention mechanism to improve the performance of 3D target detection. The algorithm utilizes point cloud data and information from the camera. For image feature extraction, the ResNet50 + FPN architecture extracts features at four levels. Point cloud feature extraction employs the voxel method and FCN to extract point and voxel features. The fusion of image and point cloud features is achieved through regional point fusion and voxel fusion methods. After information fusion, the Coordinate and SimAM attention mechanisms extract fusion features at a deep level. The algorithm's performance is evaluated using the DAIR-V2X dataset. The results show that compared to the Part-A2 algorithm; the proposed algorithm improves the mAP value by 7.9% in the BEV view and 7.8% in the 3D view at IOU = 0.5 (cars) and IOU = 0.25 (pedestrians and cyclists). At IOU = 0.7 (cars) and IOU = 0.5 (pedestrians and cyclists), the mAP value of the SECOND algorithm is improved by 5.4% in the BEV view and 4.3% in the 3D view, compared to other comparison algorithms.

An ISAR Image Component Recognition Method Based on Semantic Segmentation and Mask Matching

The inverse synthetic aperture radar (ISAR) image is a kind of target feature data acquired by radar for moving targets, which can reflect the shape, structure, and motion information of the target, and has attracted a great deal of attention from the radar automatic target recognition (RATR) community. The identification of ISAR image components in radar satellite identification missions has not been carried out in related research, and the relevant segmentation methods of optical images applied to the research of semantic segmentation of ISAR images do not achieve ideal segmentation results. To address this problem, this paper proposes an ISAR image part recognition method based on semantic segmentation and mask matching. Furthermore, a reliable automatic ISAR image component labeling method is designed, and the satellite target component labeling ISAR image samples are obtained accurately and efficiently, and the satellite target component labeling ISAR image data set is obtained. On this basis, an ISAR image component recognition method based on semantic segmentation and mask matching is proposed in this paper. U-Net and Siamese Network are designed to complete the ISAR image binary semantic segmentation and binary mask matching, respectively. The component label of the ISAR image is predicted by the mask matching results. Experiments based on satellite component labeling ISAR image datasets confirm that the proposed method is feasible and effective, and it has greater comparative advantages compared to other classical semantic segmentation networks.

Construction of Knowledge Graph of 3D Clothing Design Resources Based on Multimodal Clustering Network

The construction of 3D design model is a hotspot of applied research in the fields of clothing functional design system teaching and display. The simple 3D clothing visualization postprocessing lacks interactive functions, which is a hot issue that needs to be solved urgently at present. Based on analyzing the existing clothing modeling technology, template technology, and fusion technology, and based on the multimodal clustering network theory, this paper proposes a 3D clothing design resource knowledge graph modeling method with multiple fusion of features and templates. The position of each joint point is converted into the coordinate system centered on the torso point in advance and normalized to avoid the problem that the relative position of the camera and the collector cannot be determined, and the shape of different collectors is different. The paper provides a multimodal clustering network intelligence method, illustrates the interoperability of users switching between different design networks in the seamless connection movement, and combines the hybrid intelligence algorithm with the fuzzy logic interpretation algorithm to solve the problems in the field of 3D clothing design service quality. During the simulation process, the research scheme builds a logical multimodal clustering network framework, which integrates compatibility access and global access partition fusion of style templates to achieve information extraction of clothing parts. The experimental results show that the realistic 3D clothing modeling can be achieved by layering the 3D clothing map, contour features, clothing size features, and color texture features with the modeling template. The developed ActiveX control is mounted on MSN, and the system is compatible. The performance and integration rate reached 77.1% and 89.7%, respectively, which effectively strengthened the practical role of the 3D clothing design system.

A Complete YOLO-Based Ship Detection Method for Thermal Infrared Remote Sensing Images under Complex Backgrounds

The automatic ship detection method for thermal infrared remote sensing images (TIRSIs) is of great significance due to its broad applicability in maritime security, port management, and target searching, especially at night. Most ship detection algorithms utilize manual features to detect visible image blocks which are accurately cut, and they are limited by illumination, clouds, and atmospheric strong waves in practical applications. In this paper, a complete YOLO-based ship detection method (CYSDM) for TIRSIs under complex backgrounds is proposed. In addition, thermal infrared ship datasets were made using the SDGSAT-1 thermal imaging system. First, in order to avoid the loss of texture characteristics during large-scale deep convolution, the TIRSIs with the resolution of 30 m were up-sampled to 10 m via bicubic interpolation method. Then, complete ships with similar characteristics were selected and marked in the middle of the river, the bay, and the sea. To enrich the datasets, the gray value stretching module was also added. Finally, the improved YOLOv5 s model was used to detect the ship candidate area quickly. To reduce intra-class variation, the 4.23–7.53 aspect ratios of ships were manually selected during labeling, and 8–10.5 μm ship datasets were constructed. Test results show that the precision of the CYSDM is 98.68%, which is 9.07% higher than that of the YOLOv5s algorithm. CYSDM provides an effective reference for large-scale, all-day ship detection.

Spatial-channel relation learning for brain tumor segmentation

PURPOSE

Recently, research on brain tumor segmentation has made great progress. However, ambiguous patterns in magnetic resonance imaging data and linear fusion omitting semantic gaps between features in different branches remain challenging. We need to design a mechanism to fully utilize the similarity within the spatial space and channel space and the correlation between these two spaces to improve the result of volumetric segmentation.

METHODS

We propose a revised cascade structure network. In each subnetwork, a context exploitation module is introduced between the encoder and decoder, in which the dual attention mechanism is adopted to learn the information within the spatial space and channel space, and space interaction learning is employed to model the relation between the spatial and channel spaces.

RESULTS

Extensive experiments on the BraTS19 dataset have evaluated that our approach improves the dice coefficient (DC) by a margin of 2.1, 2.0, and 1.4 for whole tumor (WT), tumor core (TC), and enhancing tumor (ET), respectively, obtaining results competitive with the state-of-art approaches working on brain tumor segmentation.

CONCLUSIONS

Context exploitation in the embedding feature spaces, including intraspace relations and interspace relations, can effectively model dependency in semantic features and alleviate the semantic gap in multimodel data. Our approach is also robust to variations in different modality.