Deep Coupling Recurrent Auto-Encoder with Multi-Modal EEG and EOG for Vigilance Estimation

Image Geo-Site Estimation Using Convolutional Auto-Encoder and Multi-Label Support Vector Machine

The estimation of an image geo-site solely based on its contents is a promising task. Compelling image labelling relies heavily on contextual information, which is not as simple as recognizing a single object in an image. An Auto-Encode-based support vector machine approach is proposed in this work to estimate the image geo-site to address the issue of misclassifying the estimations. The proposed method for geo-site estimation is conducted using a dataset consisting of 125 classes of various images captured within 125 countries. The proposed work uses a convolutional Auto-Encode for training and dimensionality reduction. After that, the acquired preprocessed input dataset is further processed by a multi-label support vector machine. The performance assessment of the proposed approach has been accomplished using accuracy, sensitivity, specificity, and F1-score as evaluation parameters. Eventually, the proposed approach for image geo-site estimation presented in this article outperforms Auto-Encode-based K-Nearest Neighbor and Auto-Encode-Random Forest methods.

S. An image and text-based multimodal model for detecting fake news in OSN's

Digital Mass Media has become the new paradigm of communication that revolves around online social networks. The increase in the utilization of online social networks (OSNs) as the primary source of information and the increase of online social platforms providing such news has increased the scope of spreading fake news. People spread fake news in multimedia formats like images, audio, and video. Visual-based news is prone to have a psychological impact on the users and is often misleading. Therefore, Multimodal frameworks for detecting fake posts have gained demand in recent times. This paper proposes a framework that flags fake posts with Visual data embedded with text. The proposed framework works on data derived from the Fakeddit dataset, with over 1 million samples containing text, image, metadata, and comments data gathered from a wide range of sources, and tries to exploit the unique features of fake and legitimate images. The proposed framework has different architectures to learn visual and linguistic models from the post individually. Image polarity datasets, derived from Flickr, are also considered for analysis, and the features extracted from these visual and text-based data helped in flagging news. The proposed fusion model has achieved an overall accuracy of 91.94%, Precision of 93.43%, Recall of 93.07%, and F1-score of 93%. The experimental results show that the proposed Multimodality model with Image and Text achieves better results than other state-of-art models working on a similar dataset.

Effects of Paradigm Color and Screen Brightness on Visual Fatigue in Light Environment of Night Based on Eye Tracker and EEG Acquisition Equipment

Nowadays, more people tend to go to bed late and spend their sleep time with various electronic devices. At the same time, the BCI (brain−computer interface) rehabilitation equipment uses a visual display, thus it is necessary to evaluate the problem of visual fatigue to avoid the impact on the training effect. Therefore, it is very important to understand the impact of using electronic devices in a dark environment at night on human visual fatigue. This paper uses Matlab to write different color paradigm stimulations, uses a 4K display with an adjustable screen brightness to jointly design the experiment, uses eye tracker and g.tec Electroencephalogram (EEG) equipment to collect the signal, and then carries out data processing and analysis, finally obtaining the influence of the combination of different colors and different screen brightness on human visual fatigue in a dark environment. In this study, subjects were asked to evaluate their subjective (Likert scale) perception, and objective signals (pupil diameter, θ + α frequency band data) were collected in a dark environment (<3 lx). The Likert scale showed that a low screen brightness in the dark environment could reduce the visual fatigue of the subjects, and participants preferred blue to red. The pupil data revealed that visual perception sensitivity was more vulnerable to stimulation at a medium and high screen brightness, which is easier to deepen visual fatigue. EEG frequency band data concluded that there was no significant difference between paradigm colors and screen brightness on visual fatigue. On this basis, this paper puts forward a new index—the visual anti-fatigue index, which provides a valuable reference for the optimization of the indoor living environment, the improvement of satisfaction with the use of electronic equipment and BCI rehabilitation equipment, and the protection of human eyes.

Toward a Brain-Computer Interface- and Internet of Things-Based Smart Ward Collaborative System Using Hybrid Signals

This study proposes a brain-computer interface (BCI)- and Internet of Things (IoT)-based smart ward collaborative system using hybrid signals. The system is divided into hybrid asynchronous electroencephalography (EEG)-, electrooculography (EOG)- and gyro-based BCI control system and an IoT monitoring and management system. The hybrid BCI control system proposes a GUI paradigm with cursor movement. The user uses the gyro to control the cursor area selection and uses blink-related EOG to control the cursor click. Meanwhile, the attention-related EEG signals are classified based on a support-vector machine (SVM) to make the final judgment. The judgment of the cursor area and the judgment of the attention state are reduced, thereby reducing the false operation rate in the hybrid BCI system. The accuracy in the hybrid BCI control system was 96.65 ± 1.44%, and the false operation rate and command response time were 0.89 ± 0.42 events/min and 2.65 ± 0.48 s, respectively. These results show the application potential of the hybrid BCI control system in daily tasks. In addition, we develop an architecture to connect intelligent things in a smart ward based on narrowband Internet of Things (NB-IoT) technology. The results demonstrate that our system provides superior communication transmission quality.

Eye-Tracking Feature Extraction for Biometric Machine Learning

CONTEXT

Eye tracking is a technology to measure and determine the eye movements and eye positions of an individual. The eye data can be collected and recorded using an eye tracker. Eye-tracking data offer unprecedented insights into human actions and environments, digitizing how people communicate with computers, and providing novel opportunities to conduct passive biometric-based classification such as emotion prediction. The objective of this article is to review what specific machine learning features can be obtained from eye-tracking data for the classification task.

METHODS

We performed a systematic literature review (SLR) covering the eye-tracking studies in classification published from 2016 to the present. In the search process, we used four independent electronic databases which were the IEEE Xplore, the ACM Digital Library, and the ScienceDirect repositories as well as the Google Scholar. The selection process was performed by using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) search strategy. We followed the processes indicated in the PRISMA to choose the appropriate relevant articles.

RESULTS

Out of the initial 420 articles that were returned from our initial search query, 37 articles were finally identified and used in the qualitative synthesis, which were deemed to be directly relevant to our research question based on our methodology.

CONCLUSION

The features that could be extracted from eye-tracking data included pupil size, saccade, fixations, velocity, blink, pupil position, electrooculogram (EOG), and gaze point. Fixation was the most commonly used feature among the studies found.