Feature Matching of Microsecond-Pulsed Magnetic Fields Combined with Fe3O4 Particles for Killing A375 Melanoma Cells

The combination of magnetic fields and magnetic nanoparticles (MNPs) to kill cancer cells by magneto-mechanical force represents a novel therapy, offering advantages such as non-invasiveness, among others. Pulsed magnetic fields (PMFs) hold promise for application in this therapy due to advantages such as easily adjustable parameters; however, they suffer from the drawback of narrow pulse width. In order to fully exploit the potential of PMFs and MNPs in this therapy, while maximizing therapeutic efficacy within the constraints of the narrow pulse width, a feature-matching theory is proposed, encompassing the matching of three aspects: (1) MNP volume and critical volume of Brownian relaxation, (2) relaxation time and pulse width, and (3) MNP shape and the intermittence of PMF. In the theory, a microsecond-PMF generator was developed, and four kinds of MNPs were selected for in vitro cell experiments. The results demonstrate that the killing rate of the experimental group meeting the requirements of the theory is at least 18% higher than the control group. This validates the accuracy of our theory and provides valuable guidance for the further application of PMFs in this therapy.

Research on Inter-Frame Feature Mismatch Removal Method of VSLAM in Dynamic Scenes

Visual Simultaneous Localization and Mapping (VSLAM) estimates the robot's pose in three-dimensional space by analyzing the depth variations of inter-frame feature points. Inter-frame feature point mismatches can lead to tracking failure, impacting the accuracy of the mobile robot's self-localization and mapping. This paper proposes a method for removing mismatches of image features in dynamic scenes in visual SLAM. First, the Grid-based Motion Statistics (GMS) method was introduced for fast coarse screening of mismatched image features. Second, an Adaptive Error Threshold RANSAC (ATRANSAC) method, determined by the internal matching rate, was proposed to improve the accuracy of removing mismatched image features in dynamic and static scenes. Third, the GMS-ATRANSAC method was tested for removing mismatched image features, and experimental results showed that GMS-ATRANSAC can remove mismatches of image features on moving objects. It achieved an average error reduction of 29.4% and 32.9% compared to RANSAC and GMS-RANSAC, with a corresponding reduction in error variance of 63.9% and 58.0%, respectively. The processing time was reduced by 78.3% and 38%, respectively. Finally, the effectiveness of inter-frame feature mismatch removal in the initialization thread of ORB-SLAM2 and the tracking thread of ORB-SLAM3 was verified for the proposed algorithm.

Contextual Patch-NetVLAD: Context-Aware Patch Feature Descriptor and Patch Matching Mechanism for Visual Place Recognition

The goal of visual place recognition (VPR) is to determine the location of a query image by identifying its place in a collection of image databases. Visual sensor technologies are crucial for visual place recognition as they allow for precise identification and location of query images within a database. Global descriptor-based VPR methods face the challenge of accurately capturing the local specific regions within a scene; consequently, it leads to an increasing probability of confusion during localization in such scenarios. To tackle feature extraction and feature matching challenges in VPR, we propose a modified patch-NetVLAD strategy that includes two new modules: a context-aware patch descriptor and a context-aware patch matching mechanism. Firstly, we propose a context-driven patch feature descriptor to overcome the limitations of global and local descriptors in visual place recognition. This descriptor aggregates features from each patch's surrounding neighborhood. Secondly, we introduce a context-driven feature matching mechanism that utilizes cluster and saliency context-driven weighting rules to assign higher weights to patches that are less similar to densely populated or locally similar regions for improved localization performance. We further incorporate both of these modules into the patch-NetVLAD framework, resulting in a new approach called contextual patch-NetVLAD. Experimental results are provided to show that our proposed approach outperforms other state-of-the-art methods to achieve a Recall@10 score of 99.82 on Pittsburgh30k, 99.82 on FMDataset, and 97.68 on our benchmark dataset.

Robust dense-field based copy-move forgery localization using generic radial harmonic Fourier moment invariants

As the importance of reliable multimedia content increases in today's society, image forensics is a growing field of research. The act of copying and pasting specific parts of an image, known as copy-move forgery, may be utilized for illegal or unethical purposes. Just as with other vision-related technologies, the accuracy of forensic analysis depends on having an appropriate image representation. Most existing feature extraction techniques do not accurately reflect the underlying image content leading to reduced performance. In this article, to detect the copy-move forgery attack, the Generic Radial Harmonic Fourier Moment (GRHFM) is proposed for reliable and distinctive image representation. The algorithm has the ability to effectively manipulate the distribution of zeros to emphasize certain image regions. Additionally, the relationships between complex exponentials and trigonometric functions are exploited to efficiently compute and easily implement the transform kernels. The efficacy of the algorithm is illustrated through experiments on dense-domain-based matching patterns. Experimental results on five benchmarking databases prove the effectiveness of the proposed approach compared with the state-of-the-art methods. According to the average scores, the proposed method demonstrates superior accuracy in overall localization performance. The F1 score, precision, and recall percentage values obtained are 92.5, 95.44, and 91.96, respectively. Robustness experiments on more challenging attacks are also conducted on FAU dataset. Results show that the proposed framework satisfies invariance to the various image variations, and thus an enhanced robustness compared to the previous methods. Moreover, the advantage of reasonable computational cost implies its potential use in real-world forensic applications.

FILNet: Fast Image-Based Indoor Localization Using an Anchor Control Network

This paper designs a fast image-based indoor localization method based on an anchor control network (FILNet) to improve localization accuracy and shorten the duration of feature matching. Particularly, two stages are developed for the proposed algorithm. The offline stage is to construct an anchor feature fingerprint database based on the concept of an anchor control network. This introduces detailed surveys to infer anchor features according to the information of control anchors using the visual-inertial odometry (VIO) based on Google ARcore. In addition, an affine invariance enhancement algorithm based on feature multi-angle screening and supplementation is developed to solve the image perspective transformation problem and complete the feature fingerprint database construction. In the online stage, a fast spatial indexing approach is adopted to improve the feature matching speed by searching for active anchors and matching only anchor features around the active anchors. Further, to improve the correct matching rate, a homography matrix filter model is used to verify the correctness of feature matching, and the correct matching points are selected. Extensive experiments in real-world scenarios are performed to evaluate the proposed FILNet. The experimental results show that in terms of affine invariance, compared with the initial local features, FILNet significantly improves the recall of feature matching from 26% to 57% when the angular deviation is less than 60 degrees. In the image feature matching stage, compared with the initial K-D tree algorithm, FILNet significantly improves the efficiency of feature matching, and the average time of the test image dataset is reduced from 30.3 ms to 12.7 ms. In terms of localization accuracy, compared with the benchmark method based on image localization, FILNet significantly improves the localization accuracy, and the percentage of images with a localization error of less than 0.1m increases from 31.61% to 55.89%.

Enhancing Feature Detection and Matching in Low-Pixel-Resolution Hyperspectral Images Using 3D Convolution-Based Siamese Networks

Today, hyperspectral imaging plays an integral part in the remote sensing and precision agriculture field. Identifying the matching key points between hyperspectral images is an important step in tasks such as image registration, localization, object recognition, and object tracking. Low-pixel resolution hyperspectral imaging is a recent introduction to the field, bringing benefits such as lower cost and form factor compared to traditional systems. However, the use of limited pixel resolution challenges even state-of-the-art feature detection and matching methods, leading to difficulties in generating robust feature matches for images with repeated textures, low textures, low sharpness, and low contrast. Moreover, the use of narrower optics in these cameras adds to the challenges during the feature-matching stage, particularly for images captured during low-altitude flight missions. In order to enhance the robustness of feature detection and matching in low pixel resolution images, in this study we propose a novel approach utilizing 3D Convolution-based Siamese networks. Compared to state-of-the-art methods, this approach takes advantage of all the spectral information available in hyperspectral imaging in order to filter out incorrect matches and produce a robust set of matches. The proposed method initially generates feature matches through a combination of Phase Stretch Transformation-based edge detection and SIFT features. Subsequently, a 3D Convolution-based Siamese network is utilized to filter out inaccurate matches, producing a highly accurate set of feature matches. Evaluation of the proposed method demonstrates its superiority over state-of-the-art approaches in cases where they fail to produce feature matches. Additionally, it competes effectively with the other evaluated methods when generating feature matches in low-pixel resolution hyperspectral images. This research contributes to the advancement of low pixel resolution hyperspectral imaging techniques, and we believe it can specifically aid in mosaic generation of low pixel resolution hyperspectral images.

A Visual Positioning Method of UAV in a Large-Scale Outdoor Environment

Visual positioning is a basic component for UAV operation. The structure-based methods are, widely applied in most literature, based on local feature matching between a query image that needs to be localized and a reference image with a known pose and feature points. However, the existing methods still struggle with the different illumination and seasonal changes. In outdoor regions, the feature points and descriptors are similar, and the number of mismatches will increase rapidly, leading to the visual positioning becoming unreliable. Moreover, with the database growing, the image retrieval and feature matching are time-consuming. Therefore, in this paper, we propose a novel hierarchical visual positioning method, which includes map construction, landmark matching and pose calculation. First, we combine brain-inspired mechanisms and landmarks to construct a cognitive map, which can make image retrieval efficient. Second, the graph neural network is utilized to learn the inner relations of the feature points. To improve matching accuracy, the network uses the semantic confidence in matching score calculations. Besides, the system can eliminate the mismatches by analyzing all the matching results in the same landmark. Finally, we calculate the pose by using a PnP solver. Furthermore, we evaluate both the matching algorithm and the visual positioning method experimentally in the simulation datasets, where the matching algorithm performs better in some scenes. The results demonstrate that the retrieval time can be shortened by three-thirds with an average positioning error of 10.8 m.

Guidelines for Accurate Multi-Temporal Model Registration of 3D Scanned Objects

Changes in object morphology can be quantified using 3D optical scanning to generate 3D models of an object at different time points. This process requires registration techniques that align target and reference 3D models using mapping functions based on common object features that are unaltered over time. The goal of this study was to determine guidelines when selecting these localized features to ensure robust and accurate 3D model registration. For this study, an object of interest (tibia bone replica) was 3D scanned at multiple time points, and the acquired 3D models were aligned using a simple cubic registration block attached to the object. The size of the registration block and the number of planar block surfaces selected to calculate the mapping functions used for 3D model registration were varied. Registration error was then calculated as the average linear surface variation between the target and reference tibial plateau surfaces. We obtained very low target registration errors when selecting block features with an area equivalent to at least 4% of the scanning field of view. Additionally, we found that at least two orthogonal surfaces should be selected to minimize registration error. Therefore, when registering 3D models to measure multi-temporal morphological change (e.g., mechanical wear), we recommend selecting multiplanar features that account for at least 4% of the scanning field of view. For the first time, this study has provided guidelines for selecting localized object features that can provide accurate 3D model registration for 3D scanned objects.

RANSAC for Robotic Applications: A Survey

Random Sample Consensus, most commonly abbreviated as RANSAC, is a robust estimation method for the parameters of a model contaminated by a sizable percentage of outliers. In its simplest form, the process starts with a sampling of the minimum data needed to perform an estimation, followed by an evaluation of its adequacy, and further repetitions of this process until some stopping criterion is met. Multiple variants have been proposed in which this workflow is modified, typically tweaking one or several of these steps for improvements in computing time or the quality of the estimation of the parameters. RANSAC is widely applied in the field of robotics, for example, for finding geometric shapes (planes, cylinders, spheres, etc.) in cloud points or for estimating the best transformation between different camera views. In this paper, we present a review of the current state of the art of RANSAC family methods with a special interest in applications in robotics.

An Improved ASIFT Image Feature Matching Algorithm Based on POS Information

The affine scale-invariant feature transform (ASIFT) algorithm is a feature extraction algorithm with affinity and scale invariance, which is suitable for image feature matching using unmanned aerial vehicles (UAVs). However, there are many problems in the matching process, such as the low efficiency and mismatching. In order to improve the matching efficiency, this algorithm firstly simulates image distortion based on the position and orientation system (POS) information from real-time UAV measurements to reduce the number of simulated images. Then, the scale-invariant feature transform (SIFT) algorithm is used for feature point detection, and the extracted feature points are combined with the binary robust invariant scalable keypoints (BRISK) descriptor to generate the binary feature descriptor, which is matched using the Hamming distance. Finally, in order to improve the matching accuracy of the UAV images, based on the random sample consensus (RANSAC) a false matching eliminated algorithm is proposed. Through four groups of experiments, the proposed algorithm is compared with the SIFT and ASIFT. The results show that the algorithm can optimize the matching effect and improve the matching speed.

Robust Feature Matching for 3D Point Clouds with Progressive Consistency Voting

Feature matching for 3D point clouds is a fundamental yet challenging problem in remote sensing and 3D computer vision. However, due to a number of nuisances, the initial feature correspondences generated by matching local keypoint descriptors may contain many outliers (incorrect correspondences). To remove outliers, this paper presents a robust method called progressive consistency voting (PCV). PCV aims at assigning a reliable confidence score to each correspondence such that reasonable correspondences can be achieved by simply finding top-scored ones. To compute the confidence score, we suggest fully utilizing the geometric consistency cue between correspondences and propose a voting-based scheme. In addition, we progressively mine convincing voters from the initial correspondence set and optimize the scoring result by considering top-scored correspondences at the last iteration. Experiments on several standard datasets verify that PCV outperforms five state-of-the-art methods under almost all tested conditions and is robust to noise, data decimation, clutter, occlusion, and data modality change. We also apply PCV to point cloud registration and show that it can significantly improve the registration performance.

Nighttime Image Stitching Method Based on Guided Filtering Enhancement

Image stitching refers to stitching two or more images with overlapping areas through feature points matching to generate a panoramic image, which plays an important role in geological survey, military reconnaissance, and other fields. At present, the existing image stitching technologies mostly adopt images with good lighting conditions, but the lack of feature points in scenes with weak light such as morning or night will affect the image stitching effect, making it difficult to meet the needs of practical applications. When there exist concentrated areas of brightness such as lights and large dark areas in the nighttime image, it will further cause the loss of image details making the feature point matching unavailable. The obtained perspective transformation matrix cannot reflect the mapping relationship of the entire image, resulting in poor splicing effect, and it is difficult to meet the actual application requirements. Therefore, an adaptive image enhancement algorithm is proposed based on guided filtering to preprocess the nighttime image, and use the enhanced image for feature registration. The experimental results show that the image obtained by preprocessing the nighttime image with the proposed enhancement algorithm has better detail performance and color restoration, and greatly improves the image quality. By performing feature registration on the enhanced image, the number of matching logarithms of the image increases, so as to achieve high accuracy for images stitching.

Three-dimensional feature matching improves coverage for single-cell proteomics based on ion mobility filtering

Single-cell proteomics (scProteomics) promises to advance our understanding of cell functions within complex biological systems. However, a major challenge of current methods is their inability to identify and provide accurate quantitative information for low-abundance proteins. Herein, we describe an ion-mobility-enhanced mass spectrometry acquisition and peptide identification method, transferring identification based on FAIMS filtering (TIFF), to improve the sensitivity and accuracy of label-free scProteomics. TIFF extends the ion accumulation times for peptide ions by filtering out singly charged ions. The peptide identities are assigned by a three-dimensional MS1 feature matching approach (retention time, accurate mass, and FAIMS compensation voltage). The TIFF method enabled unbiased proteome analysis to a depth of >1,700 proteins in single HeLa cells, with >1,100 proteins consistently identified. As a demonstration, we applied the TIFF method to obtain temporal proteome profiles of >150 single murine macrophage cells during lipopolysaccharide stimulation and identified time-dependent proteome changes. A record of this paper's transparent peer review process is included in the supplemental information.

EMDQ: Removal of Image Feature Mismatches in Real-Time

This paper proposes a novel method for removing image feature mismatches in real-time that can handle both rigid and smooth deforming environments. Image distortion, parallax and object deformation may cause the pixel coordinates of feature matches to have non-rigid deformations, which cannot be represented using a single analytical rigid transformation. To solve this problem, we propose an algorithm based on the re-weighting and 1-point RANSAC strategy (R1P-RNSC), which operates under the assumption that a non-rigid deformation can be approximately represented by multiple rigid transformations. R1P-RNSC is fast but suffers from the drawback that local smoothing information cannot be considered, thus limiting its accuracy. To solve this problem, we propose a non-parametric algorithm based on the expectation-maximization algorithm and the dual quaternion-based representation (EMDQ). EMDQ generates dense and smooth deformation fields by interpolating among the feature matches, simultaneously removing mismatches that are inconsistent with the deformation field. It relies on the rigid transformations obtained by R1P-RNSC to improve its accuracy. The experimental results demonstrate that EMDQ has superior accuracy compared to other state-of-the-art mismatch removal methods. The ability to build correspondences for all image pixels using the dense deformation field is another contribution of this paper.

Learning Two-View Correspondences and Geometry via Local Neighborhood Correlation

Seeking quality feature correspondences (also known as matches) is a foundational step in computer vision. In our work, a novel and effective network with a stable local constraint, named the Local Neighborhood Correlation Network (LNCNet), is proposed to capture abundant contextual information of each correspondence in the local region, followed by calculating the essential matrix and camera pose estimation. Firstly, the k-Nearest Neighbor (KNN) algorithm is used to divide the local neighborhood roughly. Then, we calculate the local neighborhood correlation matrix (LNC) between the selected correspondence and other correspondences in the local region, which is used to filter outliers to obtain more accurate local neighborhood information. We cluster the filtered information into feature vectors containing richer neighborhood contextual information so that they can be used to more accurately determine the probability of correspondences as inliers. Extensive experiments have demonstrated that our proposed LNCNet performs better than some state-of-the-art networks to accomplish outlier rejection and camera pose estimation tasks in complex outdoor and indoor scenes.

LFM: A Lightweight LCD Algorithm Based on Feature Matching between Similar Key Frames

Loop Closure Detection (LCD) is an important technique to improve the accuracy of Simultaneous Localization and Mapping (SLAM). In this paper, we propose an LCD algorithm based on binary classification for feature matching between similar images with deep learning, which greatly improves the accuracy of LCD algorithm. Meanwhile, a novel lightweight convolutional neural network (CNN) is proposed and applied to the target detection task of key frames. On this basis, the key frames are binary classified according to their labels. Finally, similar frames are input into the improved lightweight feature matching network based on Transformer to judge whether the current position is loop closure. The experimental results show that, compared with the traditional method, LFM-LCD has higher accuracy and recall rate in the LCD task of indoor SLAM while ensuring the number of parameters and calculation amount. The research in this paper provides a new direction for LCD of robotic SLAM, which will be further improved with the development of deep learning.

Robust Extrinsic Calibration of Multiple RGB-D Cameras with Body Tracking and Feature Matching

RGB-D cameras have been commercialized, and many applications using them have been proposed. In this paper, we propose a robust registration method of multiple RGB-D cameras. We use a human body tracking system provided by Azure Kinect SDK to estimate a coarse global registration between cameras. As this coarse global registration has some error, we refine it using feature matching. However, the matched feature pairs include mismatches, hindering good performance. Therefore, we propose a registration refinement procedure that removes these mismatches and uses the global registration. In an experiment, the ratio of inliers among the matched features is greater than 95% for all tested feature matchers. Thus, we experimentally confirm that mismatches can be eliminated via the proposed method even in difficult situations and that a more precise global registration of RGB-D cameras can be obtained.

Dilated conditional GAN for bone suppression in chest radiographs with enforced semantic features

PURPOSE

The purpose of this essay is to improve computer-aided diagnosis of lung diseases by the removal of bone structures imagery such as ribs and clavicles, which may shadow a clinical view of lesions. This paper aims to develop an algorithm to suppress the imaging of bone structures within clinical x-ray images, leaving a residual portrayal of lung tissue; such that these images can be used to better serve applications, such as lung nodule detection or pathology based on the radiological reading of chest x rays.

METHODS

We propose a conditional Adversarial Generative Network (cGAN) (Mirza and Osindero, Conditional generative adversarial nets, 2014.) model, consisting of a generator and a discriminator, for the task of bone shadow suppression. The proposed model utilizes convolutional operations to expand the size of the receptive field of the generator without losing contextual information while downsampling the image. It is trained by enforcing both the pixel-wise intensity similarity and the semantic-level visual similarity between the generated x-ray images and the ground truth, via optimizing an L-1 loss of the pixel intensity values on the generator side and a feature matching loss on the discriminator side, respectively.

RESULTS

The framework we propose is trained and tested on an open-access chest radiograph dataset for benchmark. Results show that our model is capable of generating bone-suppressed images of outstanding quality with a limited number of training samples (N = 272).

CONCLUSIONS

Our approach outperforms current state-of-the-art bone suppression methods using x-ray images. Instead of simply downsampling images at different scales, our proposed method mitigates the loss of contextual information by utilizing dilated convolutions, which gains a noticeable quality improvement for the outputs. On the other hand, our experiment shows that enforcing the semantic similarity between the generated and the ground truth images assists the adversarial training process and achieves better perceptual quality.

A New Feature Descriptor for Multimodal Image Registration Using Phase Congruency

Images captured by different sensors with different spectral bands cause non-linear intensity changes between image pairs. Classic feature descriptors cannot handle this problem and are prone to yielding unsatisfactory results. Inspired by the illumination and contrast invariant properties of phase congruency, here, we propose a new descriptor to tackle this problem. The proposed descriptor generation mainly involves three steps. (1) Images are convolved with a bank of log-Gabor filters with different scales and orientations. (2) A window of fixed size is selected and divided into several blocks for each keypoint, and an oriented magnitude histogram and the orientation of the minimum moment of a phase congruency-based histogram are calculated in each block. (3) These two histograms are normalized respectively and concatenated to form the proposed descriptor. Performance evaluation experiments on three datasets were carried out to validate the superiority of the proposed method. Experimental results indicated that the proposed descriptor outperformed most of the classic and state-of-art descriptors in terms of precision and recall within an acceptable computational time.

Study on the Influence of Image Noise on Monocular Feature-Based Visual SLAM Based on FFDNet

Noise appears in images captured by real cameras. This paper studies the influence of noise on monocular feature-based visual Simultaneous Localization and Mapping (SLAM). First, an open-source synthetic dataset with different noise levels is introduced in this paper. Then the images in the dataset are denoised using the Fast and Flexible Denoising convolutional neural Network (FFDNet); the matching performances of Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF) and Oriented FAST and Rotated BRIEF (ORB) which are commonly used in feature-based SLAM are analyzed in comparison and the results show that ORB has a higher correct matching rate than that of SIFT and SURF, the denoised images have a higher correct matching rate than noisy images. Next, the Absolute Trajectory Error (ATE) of noisy and denoised sequences are evaluated on ORB-SLAM2 and the results show that the denoised sequences perform better than the noisy sequences at any noise level. Finally, the completely clean sequence in the dataset and the sequences in the KITTI dataset are denoised and compared with the original sequence through comprehensive experiments. For the clean sequence, the Root-Mean-Square Error (RMSE) of ATE after denoising has decreased by 16.75%; for KITTI sequences, 7 out of 10 sequences have lower RMSE than the original sequences. The results show that the denoised image can achieve higher accuracy in the monocular feature-based visual SLAM under certain conditions.

The Novel Sensor Network Structure for Classification Processing Based on the Machine Learning Method of the ACGAN

To address the problem of unstable training and poor accuracy in image classification algorithms based on generative adversarial networks (GAN), a novel sensor network structure for classification processing using auxiliary classifier generative adversarial networks (ACGAN) is proposed in this paper. Firstly, the real/fake discrimination of sensor samples in the network has been canceled at the output layer of the discriminative network and only the posterior probability estimation of the sample tag is outputted. Secondly, by regarding the real sensor samples as supervised data and the generative sensor samples as labeled fake data, we have reconstructed the loss function of the generator and discriminator by using the real/fake attributes of sensor samples and the cross-entropy loss function of the label. Thirdly, the pooling and caching method has been introduced into the discriminator to enable more effective extraction of the classification features. Finally, feature matching has been added to the discriminative network to ensure the diversity of the generative sensor samples. Experimental results have shown that the proposed algorithm (CP-ACGAN) achieves better classification accuracy on the MNIST dataset, CIFAR10 dataset and CIFAR100 dataset than other solutions. Moreover, when compared with the ACGAN and CNN classification algorithms, which have the same deep network structure as CP-ACGAN, the proposed method continues to achieve better classification effects and stability than other main existing sensor solutions.

Performance and Analysis of Feature Tracking Approaches in Laser Speckle Instrumentation

This paper investigates the application of feature tracking algorithms as an alternative data processing method for laser speckle instrumentation. The approach is capable of determining both the speckle pattern translation and rotation and can therefore be used to detect the in-plane rotation and translation of an object simultaneously. A performance assessment of widely used feature detection and matching algorithms from the computer vision field, for both translation and rotation measurements from laser speckle patterns, is presented. The accuracy of translation measurements using the feature tracking approach was found to be similar to that of correlation-based processing with accuracies of 0.025–0.04 pixels and a typical precision of 0.02–0.09 pixels depending upon the method and image size used. The performance for in-plane rotation measurements are also presented with rotation measurement accuracies of <0.01° found to be achievable over an angle range of ±10° and of <0.1° over a range of ±25°, with a typical precision between 0.02 and 0.08° depending upon method and image size. The measurement range is found to be limited by the failure to match sufficient speckles at larger rotation angles. An analysis of each stage of the process was conducted to identify the most suitable approaches for use with laser speckle images and areas requiring further improvement. A quantitative approach to assessing different feature tracking methods is described, and reference data sets of experimentally translated and rotated speckle patterns from a range of surface finishes and surface roughness are presented. As a result, three areas that lead to the failure of the matching process are identified as areas for future investigation: the inability to detect the same features in partially decorrelated images leading to unmatchable features, the variance of computed feature orientation between frames leading to different descriptors being calculated for the same feature, and the failure of the matching processes due to the inability to discriminate between different features in speckle images.

A Robust Method for Automatic Panoramic UAV Image Mosaic

This paper introduces a robust method for panoramic unmanned aerial vehicle (UAV) image mosaic. In the traditional automatic panoramic image stitching method (Autostitch), it assumes that the camera rotates about its optical centre and the group of transformations the source images may undergo is a special group of homographies. It is rare to get such ideal data in reality. In particular, remote sensing images obtained by UAV do not satisfy such an ideal situation, where the images may not be on a plane yet and even may suffer from nonrigid changes, leading to poor mosaic results. To overcome the above mentioned challenges, in this paper a nonrigid matching algorithm is introduced to the mosaic system to generate accurate feature matching on remote sensing images. We also propose a new strategy for bundle adjustment to make the mosaic system suitable for the UAV image panoramic mosaic effect. Experimental results show that our method outperforms the traditional method and some of the latest methods in terms of visual effect.

Intraoperative motion correction in neurosurgery: a comparison of intensity- and feature-based methods

The intraoperative identification of normal and anomalous brain tissue can be disturbed by pulsatile brain motion and movements of the patient and surgery devices. The performance of four motion correction methods are compared in this paper: Two intensity-based, applying optical flow algorithms, and two feature-based, which take corner features into account to track brain motion. The target registration error with manually selected marking points and the temporal standard deviation of intensity were analyzed in the evaluation. The results reveal the potential of the two types of methods.

Improved WαSH Feature Matching Based on 2D-DWT for Stereo Remote Sensing Images

Image matching is an outstanding issue because of the existing of geometric and radiometric distortion in stereo remote sensing images. Weighted α-shape (WαSH) local invariant features are tolerant to image rotation, scale change, affine deformation, illumination change, and blurring. However, since the number of WαSH features is small, it is difficult to get enough matches to estimate the satisfactory homography matrix or fundamental matrix. In addition, the WαSH detector is extremely sensitive to image noise because it is built on sampled edges. Considering the shortcomings of the WαSH detector, this paper improves the WαSH feature matching method based on the 2D discrete wavelet transform (2D-DWT). The method firstly performs 2D-DWT on the image, and then detects WαSH features on the transformed images. According to the methods of descriptor construction for WαSH features, three matching methods on the basis of wavelet transform WαSH features (WWF), improved wavelet transform WαSH features (IWWF), and layered IWWF (LIWWF) are distinguished with respect to the character of the sub-images. The experimental results on the dataset containing affine distortion, scale distortion, illumination change, and noise images, showed that the proposed methods acquired more matches and better stableness than WαSH. Experimentation on remote sensing images with less affine distortion and slight noise showed that the proposed methods obtained the correct matching rate greater than 90%. For images containing severe distortion, KAZE obtained a 35.71% correct matching rate, which is unacceptable for calculating the homography matrix, while IWWF achieved a 71.42% correct matching rate. IWWF was the only method that achieved the correct matching rate of no less than 50% for all four test stereo remote sensing image pairs and was the most stable compared to MSER, DWT-MSER, WαSH, DWT-WαSH, KAZE, WWF, and LIWWF.

Geometric Integration of Hybrid Correspondences for RGB-D Unidirectional Tracking

Traditionally, visual-based RGB-D SLAM systems only use correspondences with valid depth values for camera tracking, thus ignoring the regions without 3D information. Due to the strict limitation on measurement distance and view angle, such systems adopt only short-range constraints which may introduce larger drift errors during long-distance unidirectional tracking. In this paper, we propose a novel geometric integration method that makes use of both 2D and 3D correspondences for RGB-D tracking. Our method handles the problem by exploring visual features both when depth information is available and when it is unknown. The system comprises two parts: coarse pose tracking with 3D correspondences, and geometric integration with hybrid correspondences. First, the coarse pose tracking generates the initial camera pose using 3D correspondences with frame-by-frame registration. The initial camera poses are then used as inputs for the geometric integration model, along with 3D correspondences, 2D-3D correspondences and 2D correspondences identified from frame pairs. The initial 3D location of the correspondence is determined in two ways, from depth image and by using the initial poses to triangulate. The model improves the camera poses and decreases drift error during long-distance RGB-D tracking iteratively. Experiments were conducted using data sequences collected by commercial Structure Sensors. The results verify that the geometric integration of hybrid correspondences effectively decreases the drift error and improves mapping accuracy. Furthermore, the model enables a comparative and synergistic use of datasets, including both 2D and 3D features.

Image Registration-Based Bolt Loosening Detection of Steel Joints

Self-loosening of bolts caused by repetitive loads and vibrations is one of the common defects that can weaken the structural integrity of bolted steel joints in civil structures. Many existing approaches for detecting loosening bolts are based on physical sensors and, hence, require extensive sensor deployment, which limit their abilities to cost-effectively detect loosened bolts in a large number of steel joints. Recently, computer vision-based structural health monitoring (SHM) technologies have demonstrated great potential for damage detection due to the benefits of being low cost, easy to deploy, and contactless. In this study, we propose a vision-based non-contact bolt loosening detection method that uses a consumer-grade digital camera. Two images of the monitored steel joint are first collected during different inspection periods and then aligned through two image registration processes. If the bolt experiences rotation between inspections, it will introduce differential features in the registration errors, serving as a good indicator for bolt loosening detection. The performance and robustness of this approach have been validated through a series of experimental investigations using three laboratory setups including a gusset plate on a cross frame, a column flange, and a girder web. The bolt loosening detection results are presented for easy interpretation such that informed decisions can be made about the detected loosened bolts.

Examining sensory ability, feature matching and assessment-based adaptation for a brain-computer interface using the steady-state visually evoked potential

PURPOSE

We investigated how overt visual attention and oculomotor control influence successful use of a visual feedback brain-computer interface (BCI) for accessing augmentative and alternative communication (AAC) devices in a heterogeneous population of individuals with profound neuromotor impairments. BCIs are often tested within a single patient population limiting generalization of results. This study focuses on examining individual sensory abilities with an eye toward possible interface adaptations to improve device performance.

METHODS

Five individuals with a range of neuromotor disorders participated in four-choice BCI control task involving the steady state visually evoked potential. The BCI graphical interface was designed to simulate a commercial AAC device to examine whether an integrated device could be used successfully by individuals with neuromotor impairment.

RESULTS

All participants were able to interact with the BCI and highest performance was found for participants able to employ an overt visual attention strategy. For participants with visual deficits to due to impaired oculomotor control, effective performance increased after accounting for mismatches between the graphical layout and participant visual capabilities.

CONCLUSION

As BCIs are translated from research environments to clinical applications, the assessment of BCI-related skills will help facilitate proper device selection and provide individuals who use BCI the greatest likelihood of immediate and long term communicative success. Overall, our results indicate that adaptations can be an effective strategy to reduce barriers and increase access to BCI technology. These efforts should be directed by comprehensive assessments for matching individuals to the most appropriate device to support their complex communication needs. Implications for Rehabilitation Brain computer interfaces using the steady state visually evoked potential can be integrated with an augmentative and alternative communication device to provide access to language and literacy for individuals with neuromotor impairment. Comprehensive assessments are needed to fully understand the sensory, motor, and cognitive abilities of individuals who may use brain-computer interfaces for proper feature matching as selection of the most appropriate device including optimization device layouts and control paradigms. Oculomotor impairments negatively impact brain-computer interfaces that use the steady state visually evoked potential, but modifications to place interface stimuli and communication items in the intact visual field can improve successful outcomes.

A method to detect landmark pairs accurately between intra-patient volumetric medical images

PURPOSES

An image processing procedure was developed in this study to detect large quantity of landmark pairs accurately in pairs of volumetric medical images. The detected landmark pairs can be used to evaluate of deformable image registration (DIR) methods quantitatively.

METHODS

Landmark detection and pair matching were implemented in a Gaussian pyramid multi-resolution scheme. A 3D scale-invariant feature transform (SIFT) feature detection method and a 3D Harris-Laplacian corner detection method were employed to detect feature points, i.e., landmarks. A novel feature matching algorithm, Multi-Resolution Inverse-Consistent Guided Matching or MRICGM, was developed to allow accurate feature pairs matching. MRICGM performs feature matching using guidance by the feature pairs detected at the lower resolution stage and the higher confidence feature pairs already detected at the same resolution stage, while enforces inverse consistency.

RESULTS

The proposed feature detection and feature pair matching algorithms were optimized to process 3D CT and MRI images. They were successfully applied between the inter-phase abdomen 4DCT images of three patients, between the original and the re-scanned radiation therapy simulation CT images of two head-neck patients, and between inter-fractional treatment MRIs of two patients. The proposed procedure was able to successfully detect and match over 6300 feature pairs on average. The automatically detected landmark pairs were manually verified and the mismatched pairs were rejected. The automatic feature matching accuracy before manual error rejection was 99.4%. Performance of MRICGM was also evaluated using seven digital phantom datasets with known ground truth of tissue deformation. On average, 11855 feature pairs were detected per digital phantom dataset with TRE = 0.77 ± 0.72 mm.

CONCLUSION

A procedure was developed in this study to detect large number of landmark pairs accurately between two volumetric medical images. It allows a semi-automatic way to generate the ground truth landmark datasets that allow quantitatively evaluation of DIR algorithms for radiation therapy applications.

An Automatic Multi-Target Independent Analysis Framework for Non-Planar Infrared-Visible Registration

In this paper, we propose a novel automatic multi-target registration framework for non-planar infrared-visible videos. Previous approaches usually analyzed multiple targets together and then estimated a global homography for the whole scene, however, these cannot achieve precise multi-target registration when the scenes are non-planar. Our framework is devoted to solving the problem using feature matching and multi-target tracking. The key idea is to analyze and register each target independently. We present a fast and robust feature matching strategy, where only the features on the corresponding foreground pairs are matched. Besides, new reservoirs based on the Gaussian criterion are created for all targets, and a multi-target tracking method is adopted to determine the relationships between the reservoirs and foreground blobs. With the matches in the corresponding reservoir, the homography of each target is computed according to its moving state. We tested our framework on both public near-planar and non-planar datasets. The results demonstrate that the proposed framework outperforms the state-of-the-art global registration method and the manual global registration matrix in all tested datasets.

Improved Omnidirectional Odometry for a View-Based Mapping Approach

This work presents an improved visual odometry using omnidirectional images. The main purpose is to generate a reliable prior input which enhances the SLAM (Simultaneous Localization and Mapping) estimation tasks within the framework of navigation in mobile robotics, in detriment of the internal odometry data. Generally, standard SLAM approaches extensively use data such as the main prior input to localize the robot. They also tend to consider sensory data acquired with GPSs, lasers or digital cameras, as the more commonly acknowledged to re-estimate the solution. Nonetheless, the modeling of the main prior is crucial, and sometimes especially challenging when it comes to non-systematic terms, such as those associated with the internal odometer, which ultimately turn to be considerably injurious and compromise the convergence of the system. This omnidirectional odometry relies on an adaptive feature point matching through the propagation of the current uncertainty of the system. Ultimately, it is fused as the main prior input in an EKF (Extended Kalman Filter) view-based SLAM system, together with the adaption of the epipolar constraint to the omnidirectional geometry. Several improvements have been added to the initial visual odometry proposal so as to produce better performance. We present real data experiments to test the validity of the proposal and to demonstrate its benefits, in contrast to the internal odometry. Furthermore, SLAM results are included to assess its robustness and accuracy when using the proposed prior omnidirectional odometry.

Improved Feature Matching for Mobile Devices with IMU

Thanks to the recent diffusion of low-cost high-resolution digital cameras and to the development of mostly automated procedures for image-based 3D reconstruction, the popularity of photogrammetry for environment surveys is constantly increasing in the last years. Automatic feature matching is an important step in order to successfully complete the photogrammetric 3D reconstruction: this step is the fundamental basis for the subsequent estimation of the geometry of the scene. This paper reconsiders the feature matching problem when dealing with smart mobile devices (e.g., when using the standard camera embedded in a smartphone as imaging sensor). More specifically, this paper aims at exploiting the information on camera movements provided by the inertial navigation system (INS) in order to make the feature matching step more robust and, possibly, computationally more efficient. First, a revised version of the affine scale-invariant feature transform (ASIFT) is considered: this version reduces the computational complexity of the original ASIFT, while still ensuring an increase of correct feature matches with respect to the SIFT. Furthermore, a new two-step procedure for the estimation of the essential matrix E (and the camera pose) is proposed in order to increase its estimation robustness and computational efficiency.

Performance Analysis of the SIFT Operator for Automatic Feature Extraction and Matching in Photogrammetric Applications

In the photogrammetry field, interest in region detectors, which are widely used in Computer Vision, is quickly increasing due to the availability of new techniques. Images acquired by Mobile Mapping Technology, Oblique Photogrammetric Cameras or Unmanned Aerial Vehicles do not observe normal acquisition conditions. Feature extraction and matching techniques, which are traditionally used in photogrammetry, are usually inefficient for these applications as they are unable to provide reliable results under extreme geometrical conditions (convergent taking geometry, strong affine transformations, etc.) and for bad-textured images. A performance analysis of the SIFT technique in aerial and close-range photogrammetric applications is presented in this paper. The goal is to establish the suitability of the SIFT technique for automatic tie point extraction and approximate DSM (Digital Surface Model) generation. First, the performances of the SIFT operator have been compared with those provided by feature extraction and matching techniques used in photogrammetry. All these techniques have been implemented by the authors and validated on aerial and terrestrial images. Moreover, an auto-adaptive version of the SIFT operator has been developed, in order to improve the performances of the SIFT detector in relation to the texture of the images. The Auto-Adaptive SIFT operator (A(2) SIFT) has been validated on several aerial images, with particular attention to large scale aerial images acquired using mini-UAV systems.