A Memristive Circuit Implementation of Eyes State Detection in Fatigue Driving Based on Biological Long Short-Term Memory Rule

A dense multi-pooling convolutional network for driving fatigue detection

Driver fatigue is one of the major causes of traffic accidents, particularly for drivers of large vehicles, who are more susceptible to fatigue due to prolonged driving hours and monotonous conditions during their journeys. Existing vision-based driver fatigue detection methods fail to accurately and timely judge fatigue in complex driving scenarios (e.g., when wearing glasses or in the presence of non-driver individuals). To address these issues, this paper proposes a driving fatigue detection method based on a novel network and the analysis of driver facial actions. The proposed approach mainly consists of three submodules, i.e. Driver's State Detection (DSD), Dense Multi-Pooling Convolutional Network (DMP-Net), and Driving Fatigue Detection (DFD). In the DSD module, MTCNN is employed to locate the driver's face and detect facial landmarks in real time. Additionally, a face detection bounding box filtering algorithm is proposed to reduce false detections of the driver. To accurately detect the states of the driver's facial actions, we propose the DMP-Net network, which contains only a small number of parameters and outperforms existing methods in terms of accuracy and time consumption. The DFD module determines whether the driver is fatigued by comparing a reasonable threshold with the frequency of mouth opening (FM) and the percentage of eyelid closure over the pupil over time parameter (PERCLOS). Results of the experiments based on benchmarks and our self-collected datasets show that our method achieves 99.25% accuracy on the CEW dataset, 99.24% accuracy on the ZJU dataset, and 99.12% accuracy on our self-collected dataset. Our proposed driving fatigue detection method has as a high accuracy in real time and outperforms the existing methods.

Memristive Bionic Memory Circuit Implementation and Its Application in Multisensory Mutual Associative Learning Networks

Memory is vital and indispensable for organisms and brain-inspired intelligence to gain complete sensation and cognition of the environment. In this work, a memristive bionic memory circuit inspired by human memory model is proposed, which includes 1) receptor and sensory neuron (SN), 2) short-term memory (STM) module, and 3) long-term memory (LTM) module. By leveraging the in-memory computing characteristic of memristors, various functions such as sensation, learning, forgetting, recall, consolidation, reconsolidation, retrieval, and reset are realized. Besides, a multisensory mutual associative learning network is constructed with several bionic memory units to memorize and associate sensory information of different modalities bidirectionally. Except for association establishment, enhancement, and extinction, we also mimicked multisensory integration to manifest the synthetic process of information from different sensory channels. According to the simulation results in PSPICE, the proposed circuit performs high robustness, low area overhead, and low power consumption. Combining associative memory with human memory model, this work provides a possible idea for further research in associative learning networks.

EEG driving fatigue detection based on log-Mel spectrogram and convolutional recurrent neural networks

Driver fatigue detection is one of the essential tools to reduce accidents and improve traffic safety. Its main challenge lies in the problem of how to identify the driver's fatigue state accurately. Existing detection methods include yawning and blinking based on facial expressions and physiological signals. Still, lighting and the environment affect the detection results based on facial expressions. In contrast, the electroencephalographic (EEG) signal is a physiological signal that directly responds to the human mental state, thus reducing the impact on the detection results. This paper proposes a log-Mel spectrogram and Convolution Recurrent Neural Network (CRNN) model based on EEG to implement driver fatigue detection. This structure allows the advantages of the different networks to be exploited to overcome the disadvantages of using them individually. The process is as follows: first, the original EEG signal is subjected to a one-dimensional convolution method to achieve a Short Time Fourier Transform (STFT) and passed through a Mel filter bank to obtain a logarithmic Mel spectrogram, and then the resulting logarithmic Mel spectrogram is fed into a fatigue detection model to complete the fatigue detection task for the EEG signals. The fatigue detection model consists of a 6-layer convolutional neural network (CNN), bi-directional recurrent neural networks (Bi-RNNs), and a classifier. In the modeling phase, spectrogram features are transported to the 6-layer CNN to automatically learn high-level features, thereby extracting temporal features in the bi-directional RNN to obtain spectrogram-temporal information. Finally, the alert or fatigue state is obtained by a classifier consisting of a fully connected layer, a ReLU activation function, and a softmax function. Experiments were conducted on publicly available datasets in this study. The results show that the method can accurately distinguish between alert and fatigue states with high stability. In addition, the performance of four existing methods was compared with the results of the proposed method, all of which showed that the proposed method could achieve the best results so far.