A Deep Learning Framework for Driving Behavior Identification on In-Vehicle CAN-BUS Sensor Data

Real-time driver identification in IoV: A deep learning and cloud integration approach

The Internet of Vehicles (IoV) emerges as a pivotal extension of the Internet of Things (IoT), specifically geared towards transforming the automotive landscape. In this evolving ecosystem, the demand for a seamless end-to-end system becomes paramount for enhancing operational efficiency and safety. Hence, this study introduces an innovative method for real-time driver identification by integrating cloud computing with deep learning. Utilizing the integrated capabilities of Google Cloud, Thingsboard, and Apache Kafka, the developed solution tailored for IoV technology is adept at managing real-time data collection, processing, prediction, and visualization, with resilience against sensor data anomalies. Also, this research suggests an appropriate method for driver identification by utilizing a combination of Convolutional Neural Networks (CNN) and multi-head self-attention in the proposed approach. The proposed model is validated on two datasets: Security and collected. Moreover, the results show that the proposed model surpassed the previous works by achieving an accuracy and F1 score of 99.95%. Even when challenged with data anomalies, this model maintains a high accuracy of 96.2%. By achieving accurate driver identification results, the proposed end-to-end IoV system can aid in optimizing fleet management, vehicle security, personalized driving experiences, insurance, and risk assessment. This emphasizes its potential for road safety and managing transportation more effectively.

Design and Experimental Assessment of Real-Time Anomaly Detection Techniques for Automotive Cybersecurity

In recent decades, an exponential surge in technological advancements has significantly transformed various aspects of daily life. The proliferation of indispensable objects such as smartphones and computers underscores the pervasive influence of technology. This trend extends to the domains of the healthcare, automotive, and industrial sectors, with the emergence of remote-operating capabilities and self-learning models. Notably, the automotive industry has integrated numerous remote access points like Wi-Fi, USB, Bluetooth, 4G/5G, and OBD-II interfaces into vehicles, amplifying the exposure of the Controller Area Network (CAN) bus to external threats. With a recognition of the susceptibility of the CAN bus to external attacks, there is an urgent need to develop robust security systems that are capable of detecting potential intrusions and malfunctions. This study aims to leverage fingerprinting techniques and neural networks on cost-effective embedded systems to construct an anomaly detection system for identifying abnormal behavior in the CAN bus. The research is structured into three parts, encompassing the application of fingerprinting techniques for data acquisition and neural network training, the design of an anomaly detection algorithm based on neural network results, and the simulation of typical CAN attack scenarios. Additionally, a thermal test was conducted to evaluate the algorithm's resilience under varying temperatures.

Correlation Analysis of In-Vehicle Sensors Data and Driver Signals in Identifying Driving and Driver Behaviors

Today's cars have dozens of sensors to monitor vehicle performance through different systems, most of which communicate via vehicular networks (CAN). Many of these sensors can be used for applications other than the original ones, such as improving the driver experience or creating new safety tools. An example is monitoring variables that describe the driver's behavior. Interactions with the pedals, speed, and steering wheel, among other signals, carry driving characteristics. However, not always all variables related to these interactions are available in all vehicles; for example, the excursion of the brake pedal. Using an acquisition module, data from the in-vehicle sensors were obtained from the CAN bus, the brake pedal (externally instrumented), and the driver's signals (instrumented with an inertial sensor and electromyography of their leg), to observe the driver and car information and evaluate the correlation hypothesis between these data, as well as the importance of the brake pedal signal not usually available in all car models. Different sets of sensors were evaluated to analyze the performance of three classifiers when analyzing the driver's driving mode. It was found that there are superior results in classifying identity or behavior when driver signals are included. When the vehicle and driver attributes were used, hits above 0.93 were obtained in the identification of behavior and 0.96 in the identification of the driver; without driver signals, accuracy was more significant than 0.80 in identifying behavior. The results show a good correlation between vehicle data and data obtained from the driver, suggesting that further studies may be promising to improve the accuracy of rates based exclusively on vehicle characteristics, both for behavior identification and driver identification, thus allowing practical applications in embedded systems for local signaling and/or storing information about the driving mode, which is important for logistics companies.

Fuzzy System to Assess Dangerous Driving: A Multidisciplinary Approach

Dangerous driving can cause accidents, injuries and loss of life. An efficient assessment helps to identify the absence or degree of dangerous driving to take the appropriate decisions while driving. Previous studies assess dangerous driving through two approaches: (i) using electronic devices or sensors that provide objective variables (acceleration, turns and speed), and (ii) analyzing responses to questionnaires from behavioral science that provide subjective variables (driving thoughts, opinions and perceptions from the driver). However, we believe that a holistic and more realistic assessment requires a combination of both types of variables. Therefore, we propose a three-phase fuzzy system with a multidisciplinary (computer science and behavioral sciences) approach that draws on the strengths of sensors embedded in smartphones and questionnaires to evaluate driver behavior and social desirability. Our proposal combines objective and subjective variables while mitigating the weaknesses of the disciplines used (sensor reading errors and lack of honesty from respondents, respectively). The methods used are of proven reliability in each discipline, and their outputs feed a combined fuzzy system used to handle the vagueness of the input variables, obtaining a personalized result for each driver. The results obtained using the proposed system in a real scenario were efficient at 84.21%, and were validated with mobility experts' opinions. The presented fuzzy system can support intelligent transportation systems, driving safety, or personnel selection.

Identifying Potentially Risky Intersections for Heavy-Duty Truck Drivers Based on Individual Driving Styles

In developing countries, heavy-duty trucks play an important role in transportation for infrastructure construction. However, frequent truck accidents cause great losses. Previous studies have mainly focused on passenger drivers; to date, little has been done to assess the driving behavior of heavy truck drivers. The overall objective of this study is to classify driving styles at intersections, analyze the impacts of differing types of traffic control at intersections on driving styles, and identify potentially risky intersections. We selected 11 heavy-duty truck drivers and collected kinematic driving parameters (including driving speed and both lateral and longitudinal acceleration) from field experiments in Nanjing for our study. Our study on driving styles followed the following steps. First, we reduced data size and extracted data features on the basis of time windows in Python. Second, driving styles were classified into three driving styles: cautious, normal, and aggressive, based on the K-means clustering method, and the corresponding thresholds for each category were obtained. Kinematic driving parameters were used as driving style measurements. Third, according to classifications of driving style, the impacts of four different intersection traffic control types: two-phase signalized, multiphase signalized, stop, and yield intersections, on driving styles have been analyzed using the multinomial logit model. Moreover, based on the above analysis, potentially risky intersections were identified. The results suggest that different types of traffic control at intersections lead to variations in driving styles and have different influences on driving styles. In terms of accuracy, our method, which uses driving speed, both lateral and longitudinal acceleration, and jerk as features, performs better than traditional methods which only use speed and acceleration. The results of the study allow us to analyze the driving data of heavy-duty trucks and identify drivers who drive more aggressively during a trip. In addition, the results show that aggressive driving styles mostly occur at stop intersections and in the dilemma zones of signalized intersections. Therefore, early-warning interventions can be provided during a driver’s trip by analyzing the different types of traffic control at intersections on the route in advance. Finally, the cumulative analysis of driving styles at intersections over multiple trips can be used to identify potentially high-risk intersections. It is possible to eliminate potential risks in these areas through measures such as early warnings and by improving traffic management control methods.

TCAN-IDS: Intrusion Detection System for Internet of Vehicle Using Temporal Convolutional Attention Network

Intrusion detection systems based on recurrent neural network (RNN) have been considered as one of the effective methods to detect time-series data of in-vehicle networks. However, building a model for each arbitration bit is not only complex in structure but also has high computational overhead. Convolutional neural network (CNN) has always performed excellently in processing images, but they have recently shown great performance in learning features of normal and attack traffic by constructing message matrices in such a manner as to achieve real-time monitoring but suffer from the problem of temporal relationships in context and inadequate feature representation in key regions. Therefore, this paper proposes a temporal convolutional network with global attention to construct an in-vehicle network intrusion detection model, called TCAN-IDS. Specifically, the TCAN-IDS model continuously encodes 19-bit features consisting of an arbitration bit and data field of the original message into a message matrix, which is symmetric to messages recalling a historical moment. Thereafter, the feature extraction model extracts its spatial-temporal detail features. Notably, global attention enables global critical region attention based on channel and spatial feature coefficients, thus ignoring unimportant byte changes. Finally, anomalous traffic is monitored by a two-class classification component. Experiments show that TCAN-IDS demonstrates high detection performance on publicly known attack datasets and is able to accomplish real-time monitoring. In particular, it is anticipated to provide a high level of symmetry between information security and illegal intrusion.

Development of deep learning-based detecting systems for pathologic myopia using retinal fundus images

Globally, cases of myopia have reached epidemic levels. High myopia and pathological myopia (PM) are the leading cause of visual impairment and blindness in China, demanding a large volume of myopia screening tasks to control the rapid growing myopic prevalence. It is desirable to develop the automatically intelligent system to facilitate these time- and labor- consuming tasks. In this study, we designed a series of deep learning systems to detect PM and myopic macular lesions according to a recent international photographic classification system (META-PM) classification based on color fundus images. Notably, our systems recorded robust performance both in the test and external validation dataset. The performance was comparable to the general ophthalmologist and retinal specialist. With the extensive adoption of this technology, effective mass screening for myopic population will become feasible on a national scale.

Lu et al. develop a deep-learning based detection system for identifying pathological myopia and myopic macular legions in retinal fundus images. The system performance is comparable to human experts, but much faster, easing the burden of human time on screening for myopia.

GSS-RiskAsser: A Multi-Modal Deep-Learning Framework for Urban Gas Supply System Risk Assessment on Business Users

Gas supply system risk assessment is a serious and important problem in cities. Existing methods tend to manually build mathematical models to predict risk value from single-modal information, i.e., pipeline parameters. In this paper, we attempt to consider this problem from a deep-learning perspective and define a novel task, Urban Gas Supply System Risk Assessment (GSS-RA). To drive deep-learning techniques into this task, we collect and build a domain-specific dataset GSS-20K containing multi-modal data. Accompanying the dataset, we design a new deep-learning framework named GSS-RiskAsser to learn risk prediction. In our method, we design a parallel-transformers Vision Embedding Transformer (VET) and Score Matrix Transformer (SMT) to process multi-modal information, and then propose a Multi-Modal Fusion (MMF) module to fuse the features with a cross-attention mechanism. Experiments show that GSS-RiskAsser could work well on GSS-RA task and facilitate practical applications. Our data and code will be made publicly available.

Prediction of Long-Term Elbow Flexion Force Intervals Based on the Informer Model and Electromyography

Accurate and long-term prediction of elbow flexion force can be used to recognize the intended movement and help wearable power-assisted robots to improve control performance. Our study aimed to find a proper relationship between electromyography and flexion force. However, the existing methods must incorporate biomechanical models to produce accurate and timely predictions of flexion force. Elbow flexion force is largely determined by the contractile properties of muscles, and the relationship between flexion force and the motor function of muscles has to be thoroughly analyzed. Therefore, based on the investigation on the contributions of different muscles to the flexion force, original electromyography signals were decomposed into non-linear and non-stationary parts. We selected the mean absolute value (MAV) of the non-linear part and the variance of the non-stationary part as inputs for an Informer prediction model that does not require detailed a priori knowledge of biomechanical models and is optimized for processing time sequences. Finally, a long-term flexion force probability interval is proposed. The proposed framework performs well in predicting long-term flexion force and outperforms other state-of-the-art models when compared to experimental results.

A Distributed Edge-Based Scheduling Technique with Low-Latency and High-Bandwidth for Existing Driver Profiling Algorithms

The gradual increase in latency-sensitive, real-time applications for embedded systems encourages users to share sensor data simultaneously. Streamed sensor data have deficient performance. In this paper, we propose a new edge-based scheduling method with high-bandwidth for decreasing driver-profiling latency. The proposed multi-level memory scheduling method places data in a key-value storage, flushes sensor data when the edge memory is full, and reduces the number of I/O operations, network latency, and the number of REST API calls in the edge cloud. As a result, the proposed method provides significant read/write performance enhancement for real-time embedded systems. In fact, the proposed application improves the number of requests per second by 3.5, 5, and 4 times, respectively, compared with existing light-weight FCN-LSTM, FCN-LSTM, and DeepConvRNN Attention solutions. The proposed application also improves the bandwidth by 5.89, 5.58, and 4.16 times respectively, compared with existing light-weight FCN-LSTM, FCN-LSTM, and DeepConvRNN Attention solutions.

Improving Machine Learning Identification of Unsafe Driver Behavior by Means of Sensor Fusion

Most road accidents occur due to human fatigue, inattention, or drowsiness. Recently, machine learning technology has been successfully applied to identifying driving styles and recognizing unsafe behaviors starting from in-vehicle sensors signals such as vehicle and engine speed, throttle position, and engine load. In this work, we investigated the fusion of different external sensors, such as a gyroscope and a magnetometer, with in-vehicle sensors, to increase machine learning identification of unsafe driver behavior. Starting from those signals, we computed a set of features capable to accurately describe the behavior of the driver. A support vector machine and an artificial neural network were then trained and tested using several features calculated over more than 200 km of travel. The ground truth used to evaluate classification performances was obtained by means of an objective methodology based on the relationship between speed, and lateral and longitudinal acceleration of the vehicle. The classification results showed an average accuracy of about 88% using the SVM classifier and of about 90% using the neural network demonstrating the potential capability of the proposed methodology to identify unsafe driver behaviors.

Lightweight Driver Behavior Identification Model with Sparse Learning on In-Vehicle CAN-BUS Sensor Data

This study focuses on driver-behavior identification and its application to finding embedded solutions in a connected car environment. We present a lightweight, end-to-end deep-learning framework for performing driver-behavior identification using in-vehicle controller area network (CAN-BUS) sensor data. The proposed method outperforms the state-of-the-art driver-behavior profiling models. Particularly, it exhibits significantly reduced computations (i.e., reduced numbers both of floating-point operations and parameters), more efficient memory usage (compact model size), and less inference time. The proposed architecture features depth-wise convolution, along with augmented recurrent neural networks (long short-term memory or gated recurrent unit), for time-series classification. The minimum time-step length (window size) required in the proposed method is significantly lower than that required by recent algorithms. We compared our results with compressed versions of existing models by applying efficient channel pruning on several layers of current models. Furthermore, our network can adapt to new classes using sparse-learning techniques, that is, by freezing relatively strong nodes at the fully connected layer for the existing classes and improving the weaker nodes by retraining them using data regarding the new classes. We successfully deploy the proposed method in a container environment using NVIDIA Docker in an embedded system (Xavier, TX2, and Nano) and comprehensively evaluate it with regard to numerous performance metrics.

Complementary Deep and Shallow Learning with Boosting for Public Transportation Safety

To monitor road safety, billions of records can be generated by Controller Area Network bus each day on public transportation. Automation to determine whether certain driving behaviour of drivers on public transportation can be considered safe on the road using artificial intelligence or machine learning techniques for big data analytics has become a possibility recently. Due to the high false classification rates of the current methods, our goal is to build a practical and accurate method for road safety predictions that automatically determine if the driving behaviour is safe on public transportation. In this paper, our main contributions include (1) a novel feature extraction method because of the lack of informative features in raw CAN bus data, (2) a novel boosting method for driving behaviour classification (safe or unsafe) to combine advantages of deep learning and shallow learning methods with much improved performance, and (3) an evaluation of our method using a real-world data to provide accurate labels from domain experts in the public transportation industry for the first time. The experiments show that the proposed boosting method with our proposed features outperforms seven other popular methods on the real-world dataset by 5.9% and 5.5%.