When Dijkstra Meets Vanishing Point: A Stereo Vision Approach for Road Detection

StructLane: Leveraging Structural Relations for Lane Detection

Accurately detecting the lanes plays a significant role in various autonomous and assistant driving scenarios. It is a highly structured task as lanes in the 3D world are continuous and parallel to each other. While most existing methods focus on how to inject structural priors into the representation of each lane, we propose a StructLane method to further leverage the structural relations among lanes for more accurate and robust lane detection. To achieve this, we explicitly encode the structural relations using a set of relational templates in a learned structural space. We then employ the attention mechanism to enable interactions between templates and image features to incorporate structural relational priors. Our StructLane can be applied to existing lane detection methods as a plug-and-play module to improve their performance. Extensive experiments on the widely used CULane, TuSimple, and LLAMAS datasets demonstrate that StructLane consistently improves the performance of state-of-the-art models across all datasets and backbones. Visualization results also demonstrate the robustness of our StructLane compared with existing methods due to the leverage of structural relations. Codes will be released at https://github.com/lqzhao/StructLane.

Automated algorithm aided capacity and confidence boost in surgical decision-making training for inferior clivus

Objective

To assess the impact of automated algorithms on the trainees' decision-making capacity and confidence for individualized surgical planning.

Methods

At Chinese PLA General Hospital, trainees were enrolled to undergo decision-making capacity and confidence training through three alternative visual tasks of the inferior clivus model formed from an automated algorithm and given consecutively in three exemplars. The rationale of automated decision-making was used to instruct each trainee.

Results

Following automated decision-making calculation in 50 skull base models, we screened out three optimal plans, infra-tubercle approach (ITA), trans-tubercle approach (TTA), and supra-tubercle approach (STA) for 41 (82.00%), 8 (16.00%), and 1 (2.00%) subject, respectively. From September 1, 2023, through November 17, 2023, 62 trainees (median age [range]: 27 [26-28]; 28 [45.16%] female; 25 [40.32%] neurosurgeons) made a decision among the three plans for the three typical models (ITA, TTA, and STA exemplars). The confidence ratings had fine test-retest reliability (Spearman's rho: 0.979; 95% CI: 0.970 to 0.988) and criterion validity with time spent (Spearman's rho: -0.954; 95%CI: -0.963 to -0.945). Following instruction of automated decision-making, time spent (initial test: 24.02 vs. 7.13 in ITA; 30.24 vs. 7.06 in TTA; 34.21 vs. 12.82 in STA) and total hits (initial test: 30 vs. 16 in ITA; 37 vs. 17 in TTA; 42 vs. 28 in STA) reduced significantly; confidence ratings (initial test: 2 vs. 4 in ITA; 2 vs. 4 in TTA; 1 vs. 3 in STA) increased correspondingly. Statistically significant differences (P < 0.05) were observed for the above comparisons.

Conclusions

The education tool generated by automated decision-making considers surgical freedom and injury risk for the individualized risk-benefit assessment, which may provide explicit information to increase trainees' decision-making capacity and confidence.

Detail Preserving Coarse-to-Fine Matching for Stereo Matching and Optical Flow

The Coarse-To-Fine (CTF) matching scheme has been widely applied to reduce computational complexity and matching ambiguity in stereo matching and optical flow tasks by converting image pairs into multi-scale representations and performing matching from coarse to fine levels. Despite its efficiency, it suffers from several weaknesses, such as tending to blur the edges and miss small structures like thin bars and holes. We find that the pixels of small structures and edges are often assigned with wrong disparity/flow in the upsampling process of the CTF framework, introducing errors to the fine levels and leading to such weaknesses. We observe that these wrong disparity/flow values can be avoided if we select the best-matched value among their neighborhood, which inspires us to propose a novel differentiable Neighbor-Search Upsampling (NSU) module. The NSU module first estimates the matching scores and then selects the best-matched disparity/flow for each pixel from its neighbors. It effectively preserves finer structure details by exploiting the information from the finer level while upsampling the disparity/flow. The proposed module can be a drop-in replacement of the naive upsampling in the CTF matching framework and allows the neural networks to be trained end-to-end. By integrating the proposed NSU module into a baseline CTF matching network, we design our Detail Preserving Coarse-To-Fine (DPCTF) matching network. Comprehensive experiments demonstrate that our DPCTF can boost performances for both stereo matching and optical flow tasks. Notably, our DPCTF achieves new state-of-the-art performances for both tasks - it outperforms the competitive baseline (Bi3D) by 28.8% (from 0.73 to 0.52) on EPE of the FlyingThings3D stereo dataset, and ranks first in KITTI flow 2012 benchmark. The code is available at https://github.com/Deng-Y/DPCTF.

A Robust Road Vanishing Point Detection Adapted to the Real-world Driving Scenes

Vanishing point (VP) provides extremely useful information related to roads in driving scenes for advanced driver assistance systems (ADAS) and autonomous vehicles. Existing VP detection methods for driving scenes still have not achieved sufficiently high accuracy and robustness to apply for real-world driving scenes. This paper proposes a robust motion-based road VP detection method to compensate for the deficiencies. For such purposes, three main processing steps often used in the existing road VP detection methods are carefully examined. Based on the analysis, stable motion detection, stationary point-based motion vector selection, and angle-based RANSAC (RANdom SAmple Consensus) voting are proposed. A ground-truth driving dataset including various objects and illuminations is used to verify the robustness and real-time capability of the proposed method. The experimental results show that the proposed method outperforms the existing motion-based and edge-based road VP detection methods for various illumination conditioned driving scenes.

Efficient Vanishing Point Detection for Driving Assistance Based on Visual Saliency Map and Image Segmentation from a Vehicle Black-Box Camera

Techniques for detecting a vanishing point (VP) which estimates the direction of a vehicle by analyzing its relationship with surrounding objects have gained considerable attention recently. VPs can be used to support safe vehicle driving in areas such as for autonomous driving, lane-departure avoidance, distance estimation, and road-area detection, by detecting points in which parallel extension lines of objects are concentrated at a single point in a 3D space. In this paper, we proposed a method of detecting the VP in real time for applications to intelligent safe-driving support systems. In order to support safe driving of autonomous vehicles, it is necessary to drive the vehicle with the VP in center of the road image in order to prevent the vehicle from moving out of the road area while driving. Accordingly, in order to detect the VP in the road image, a method of detecting a point where straight lines intersect in an area where edge directional feature information is concentrated is required. The visual attention model and image segmentation process are applied to quickly identify candidate VPs in the area where the edge directional feature-information is concentrated and the intensity contrast difference is large. In the proposed method, VPs are detected by analyzing the edges, visual-attention regions, linear components using the Hough transform, and image segmentation results in an input image. Our experimental results have shown that the proposed method could be applied to safe-driving support systems.

RESLS: Region and Edge Synergetic Level Set Framework for Image Segmentation

The active contour models with level set evolution have been visited with a vast number of methods for image segmentation. They can be mainly classified into region-based and edge-based models, and it has been validated that the hybrid variants combining both region and edge information can improve the segmentation performance. However, to the best of our knowledge, the theoretical foundation of collaboration mechanism between the region and the edge information is limited. Specifically, most existing hybrid models are just combining all the energy terms together, resulting in great challenges of choosing an appropriate weight coefficient for each term and accommodating different modalities of imaging. To overcome these difficulties, this paper proposes a region and edge synergetic level set framework named RESLS. It provides an approach to construct new hybrid level set models using a normalized intensity indicator function that allows the region information easily embedding into the edge-based model. In this case, the energy weights of region and edge terms can be constrained by the global optimization condition deduced from the framework. Some representative as well as state-of-the-art models are taken as examples to demonstrate the generality of our method. The experiments validate that under the guidance of the optimization condition, the weighting parameter of each term can be reliably chosen. Meanwhile, the segmentation accuracy, robustness, and computational efficiency of RESLS can be improved compared with its component models.

A real-time road detection method based on reorganized lidar data

Road Detection is a basic task in automated driving field, in which 3D lidar data is commonly used recently. In this paper, we propose to rearrange 3D lidar data into a new organized form to construct direct spatial relationship among point cloud, and put forward new features for real-time road detection tasks. Our model works based on two prerequisites: (1) Road regions are always flatter than non-road regions. (2) Light travels in straight lines in a uniform medium. Based on prerequisite 1, we put forward difference-between-lines feature, while ScanID density and obstacle radial map are generated based on prerequisite 2. According to our method, we construct an array of structures to store and reorganize 3D input firstly. Then, two novel features, difference-between-lines and ScanID density, are extracted, based on which we construct a consistency map and an obstacle map in Bird Eye View (BEV). Finally, the road region is extracted by fusing these two maps and refinement is used to polish up our outcome. We have carried out experiments on the public KITTI-Road benchmark, achieving one of the best performances among the lidar-based road detection methods. To further prove the efficiency of our method on unstructured road, the visual outcomes in rural areas are also proposed.

Online Road Detection under a Shadowy Traffic Image Using a Learning-Based Illumination-Independent Image

Shadows and normal light illumination and road and non-road areas are two pairs of contradictory symmetrical individuals. To achieve accurate road detection, it is necessary to remove interference caused by uneven illumination, such as shadows. This paper proposes a road detection algorithm based on a learning and illumination-independent image to solve the following problems: First, most road detection methods are sensitive to variation of illumination. Second, with traditional road detection methods based on illumination invariability, it is difficult to determine the calibration angle of the camera axis, and the sampling of road samples can be distorted. The proposed method contains three stages: The establishment of a classifier, the online capturing of an illumination-independent image, and the road detection. During the establishment of a classifier, a support vector machine (SVM) classifier for the road block is generated through training with the multi-feature fusion method. During the online capturing of an illumination-independent image, the road interest region is obtained by using a cascaded Hough transform parameterized by a parallel coordinate system. Five road blocks are obtained through the SVM classifier, and the RGB (Red, Green, Blue) space of the combined road blocks is converted to a geometric mean log chromatic space. Next, the camera axis calibration angle for each frame is determined according to the Shannon entropy so that the illumination-independent image of the respective frame is obtained. During the road detection, road sample points are extracted with the random sampling method. A confidence interval classifier of the road is established, which could separate a road from its background. This paper is based on public datasets and video sequences, which records roads of Chinese cities, suburbs, and schools in different traffic scenes. The author compares the method proposed in this paper with other sound video-based road detection methods and the results show that the method proposed in this paper can achieve a desired detection result with high quality and robustness. Meanwhile, the whole detection system can meet the real-time processing requirement.