RefineFace: Refinement Neural Network for High Performance Face Detection

Face mask detection and social distance monitoring system for COVID-19 pandemic

Coronavirus triggers several respirational infections such as sneezing, coughing, and pneumonia, which transmit humans to humans through airborne droplets. According to the guidelines of the World Health Organization, the spread of COVID-19 can be mitigated by avoiding public interactions in proximity and following standard operating procedures (SOPs) including wearing a face mask and maintaining social distancing in schools, shopping malls, and crowded areas. However, enforcing the adaptation of these SOPs on a larger scale is still a challenging task. With the emergence of deep learning-based visual object detection networks, numerous methods have been proposed to perform face mask detection on public spots. However, these methods require a huge amount of data to ensure robustness in real-time applications. Also, to the best of our knowledge, there is no standard outdoor surveillance-based dataset available to ensure the efficacy of face mask detection and social distancing methods in public spots. To this end, we present a large-scale dataset comprising of 10,000 outdoor images categorized into a binary class labeling i.e., face mask, and non-face masked people to accelerate the development of automated face mask detection and social distance measurement on public spots. Alongside, we also present an end-to-end pipeline to perform real-time face mask detection and social distance measurement in an outdoor environment. Initially, existing state-of-the-art single and multi-stage object detection networks are fine-tuned on the proposed dataset to evaluate their performance in terms of accuracy and inference time. Based on better performance, YOLO-v3 architecture is further optimized by tuning its feature extraction and region proposal generation layers to improve the performance in real-time applications. Our results indicate that the presented pipeline performed better than the baseline version, showing an improvement of 5.3% in terms of accuracy.

Wide aspect ratio matching for robust face detection

Recently, anchor-based methods have achieved great progress in face detection. They adopt standard anchor matching strategy to sample positive anchors according to predefined IoU threshold. However, the max IoUs of extreme aspect ratio faces are still lower than fixed positive threshold, leading to the sampling failure from these faces. To construct a more robust detection model, more positive anchors from extreme aspect ratio faces need to be sampled and participate in the training phase. The goal of the present research is to improve the detection performance by reasonably extending sampling range of face aspect ratio. In this paper, we firstly explore the factors that affect the max IoU of each face in theory. Then, anchor matching simulation is performed to evaluate the sampling range of face aspect ratio. Finally, we propose a Wide Aspect Ratio Matching (WARM) strategy to collect more representative positive anchors from ground-truth faces across a wide range of aspect ratios. Besides, we present a novel feature enhancement module, named Receptive Field Diversity (RFD) module, to provide diverse receptive field corresponding to different aspect ratios. Extensive experiments have been conducted on popular benchmarks to show the effectiveness of our method, which can help detectors better capture extreme aspect ratio faces. Our method achieves promising APs on WIDER FACE validation dataset (easy: 0.965, medium: 0.955, hard: 0.904) and impressive generalization capability on FDDB dataset.

A Computer Vision Model to Identify the Incorrect Use of Face Masks for COVID-19 Awareness

Face mask detection has become a great challenge in computer vision, demanding the coalition of technology with COVID-19 awareness. Researchers have proposed deep learning models to detect the use of face masks. However, the incorrect use of a face mask can be as harmful as not wearing any protection at all. In this paper, we propose a compound convolutional neural network (CNN) architecture based on two computer vision tasks: object localization to discover faces in images/videos, followed by an image classification CNN to categorize the faces and show if someone is using a face mask correctly, incorrectly, or not at all. The first CNN is built upon RetinaFace, a model to detect faces in images, whereas the second CNN uses a ResNet-18 architecture as a classification backbone. Our model enables an accurate identification of people who are not correctly following the COVID-19 healthcare recommendations on face mask use. To enable further global use of our technology, we have released both the dataset used to train the classification model and our proposed computer vision pipeline to the public, and optimized it for embedded systems deployment.

How head posture affects perceived cooperativeness: A cross-cultural perspective

Previous research has tested whether culture moderates the relationship between head tilt and perceptions of a cooperation-relevant construct. In this paper, we replicated the effects of head posture on perceived traits and compared Chinese and American participants to explore whether difference in cultural background (collectivist and individualist) affects perceptual attribution. Specifically, we investigated how head posture (level, up or down) affects perceptions of cooperativeness. In Experiment 1, Chinese and American participants rated Asian and Caucasian faces in three postures for perceived cooperativeness on a seven-point Likert scale. In Experiment 2, participants ranked the cooperativeness of the three postures of the same faces. In Experiment 3, participants scrolled through face images and manually manipulated vertical head angle to maximise apparent cooperativeness. We found that for both Chinese and American participants a neutral head level posture was perceived as more cooperative than head up and down postures. The optimal head posture for maximised apparent cooperativeness was close to level but with a slight downward rotation. While there was cross-cultural consistency in perceptions, Chinese participants exhibited greater sensitivity to postural cues in their judgments of cooperation compared to American participants. Our results suggest a profound effect of posture on the perception of cooperativeness that is common across cultures and that there are additional subtle cross-cultural differences in the cues to cooperativeness.

FAFD: Fast and Accurate Face Detector

Deep Neural Networks (DNN) have contributed a significant performance improvement in face detection. However, since most models focus only on the improvement of detection accuracy with computationally expensive structures, it is still far from real-time applications with a fast face detector. The goal of this paper is to improve face detection performance from the speed-focusing point of view. To this end, we propose a novel Fast and Accurate Face Detector (FAFD) to achieve high performance on both speed and accuracy performance. Specifically, based on the YOLOv5 model, we add one prediction head to increase the detection performance, especially for small faces. In addition, to increase the detection performance of multi-scale faces, we propose to add a novel Multi-Scale Image Fusion (MSIF) layer to the backbone network. We also propose an improved Copy-Paste to augment the training images with face objects in various scales. Experimental results on the WiderFace dataset show that the proposed FAFD achieves the best performance among the existing methods in a Speed-Focusing group. On three sub-datasets of WiderFace (i.e., Easy, Medium, and Hard sub-datasets), our FAFD yields average precisions (AP) of 95.0%, 93.5%, and 87.0%, respectively. Also, the speed performance of the FAFD is fast enough to be included in the group of speed-focusing methods.

Object Detection and Classification Based on YOLO-V5 with Improved Maritime Dataset

SMD (Singapore Maritime Dataset) is a public dataset with annotated videos, and it is almost unique in the training of deep neural networks (DNN) for the recognition of maritime objects. However, there are noisy labels and imprecisely located bounding boxes in the ground truth of the SMD. In this paper, for the benchmark of DNN algorithms, we correct the annotations of the SMD dataset and present an improved version, which we coined SMD-Plus. We also propose augmentation techniques designed especially for the SMD-Plus. More specifically, an online transformation of training images via Copy & Paste is applied to solve the class-imbalance problem in the training dataset. Furthermore, the mix-up technique is adopted in addition to the basic augmentation techniques for YOLO-V5. Experimental results show that the detection and classification performance of the modified YOLO-V5 with the SMD-Plus has improved in comparison to the original YOLO-V5. The ground truth of the SMD-Plus and our experimental results are available for download.

SF-CNN: Signal Filtering Convolutional Neural Network for Precipitation Intensity Estimation

Precipitation intensity estimation is a critical issue in the analysis of weather conditions. Most existing approaches focus on building complex models to extract rain streaks. However, an efficient approach to estimate the precipitation intensity from surveillance cameras is still challenging. This study proposes a convolutional neural network known as the signal filtering convolutional neural network (SF-CNN) to handle precipitation intensity using surveillance-based images. The SF-CNN has two main blocks, the signal filtering block (SF block) and the gradually decreasing dimension block (GDD block), to extract features for the precipitation intensity estimation. The SF block with the filtering operation is constructed in different parts of the SF-CNN to remove the noise from the features containing rain streak information. The GDD block continuously takes the pair of the convolutional operation with the activation function to reduce the dimension of features. Our main contributions are (1) an SF block considering the signal filtering process and effectively removing the useless signals and (2) a procedure of gradually decreasing the dimension of the feature able to learn and reserve the information of features. Experiments on the self-collected dataset, consisting of 9394 raining images with six precipitation intensity levels, demonstrate the proposed approach’s effectiveness against the popular convolutional neural networks. To the best of our knowledge, the self-collected dataset is the largest dataset for monitoring infrared images of precipitation intensity.

Efficient Face Recognition System for Operating in Unconstrained Environments

Facial recognition is fundamental for a wide variety of security systems operating in real-time applications. Recently, several deep neural networks algorithms have been developed to achieve state-of-the-art performance on this task. The present work was conceived due to the need for an efficient and low-cost processing system, so a real-time facial recognition system was proposed using a combination of deep learning algorithms like FaceNet and some traditional classifiers like SVM, KNN, and RF using moderate hardware to operate in an unconstrained environment. Generally, a facial recognition system involves two main tasks: face detection and recognition. The proposed scheme uses the YOLO-Face method for the face detection task which is a high-speed real-time detector based on YOLOv3, while, for the recognition stage, a combination of FaceNet with a supervised learning algorithm, such as the support vector machine (SVM), is proposed for classification. Extensive experiments on unconstrained datasets demonstrate that YOLO-Face provides better performance when the face under an analysis presents partial occlusion and pose variations; besides that, it can detect small faces. The face detector was able to achieve an accuracy of over 89.6% using the Honda/UCSD dataset which runs at 26 FPS with darknet-53 to VGA-resolution images for classification tasks. The experimental results have demonstrated that the FaceNet+SVM model was able to achieve an accuracy of 99.7% using the LFW dataset. On the same dataset, FaceNet+KNN and FaceNet+RF achieve 99.5% and 85.1%, respectively; on the other hand, the FaceNet was able to achieve 99.6%. Finally, the proposed system provides a recognition accuracy of 99.1% and 49 ms runtime when both the face detection and classifications stages operate together.

Efficient Face Detection in the Fisheye Image Domain

Significant progress has been made for face detection from normal images in recent years; however, accurate and fast face detection from fisheye images remains a challenging issue because of serious fisheye distortion in the peripheral region of the image. To improve face detection accuracy, we propose a light-weight location-aware network to distinguish the peripheral region from the central region in the feature learning stage. To match the face detector, the shape and scale of the anchor (bounding box) is made location dependent. The overall face detection system performs directly in the fisheye image domain without rectification and calibration and hence is agnostic of the fisheye projection parameters. Experiments on Wider-360 and real-world fisheye images using a single CPU core indeed show that our method is superior to the state-of-the-art real-time face detector RFB Net.

Secure Autonomous Cloud Brained Humanoid Robot Assisting Rescuers in Hazardous Environments

On 31 January 2020, the World Health Organization (WHO) declared a global emergency after the discovery of a new pandemic disease that caused severe lung problems. The spread of the disease at an international level drew the attention of many researchers who attempted to find solutions to ameliorate the problem. The implementation of robotics has been one of the proposed solutions, as automated humanoid robots can be used in many situations and limit the exposure of humans to the disease. Many humanoid robot implementations are found in the literature; however, most of them have some distinct drawbacks, such as a high cost and complexity. Our research proposes a novel, secure and efficient programmable system using a humanoid robot that is able to autonomously move and detect survivors in emergency scenarios, with the potential to communicate verbally with victims. The proposed humanoid robot is powered by the cloud and benefits from the powerful storage, computation, and communication resources of a typical modern data center. In order to evaluate the proposed system, we conducted multiple experiments in synthetic hazardous environments.

A Gesture Elicitation Study of Nose-Based Gestures

Presently, miniaturized sensors can be embedded in any small-size wearable to recognize movements on some parts of the human body. For example, an electrooculography-based sensor in smart glasses recognizes finger movements on the nose. To explore the interaction capabilities, this paper conducts a gesture elicitation study as a between-subjects experiment involving one group of 12 females and one group of 12 males, expressing their preferred nose-based gestures on 19 Internet-of-Things tasks. Based on classification criteria, the 912 elicited gestures are clustered into 53 unique gestures resulting in 23 categories, to form a taxonomy and a consensus set of 38 final gestures, providing researchers and practitioners with a larger base with six design guidelines. To test whether the measurement method impacts these results, the agreement scores and rates, computed for determining the most agreed gestures upon participants, are compared with the Condorcet and the de Borda count methods to observe that the results remain consistent, sometimes with a slightly different order. To test whether the results are sensitive to gender, inferential statistics suggest that no significant difference exists between males and females for agreement scores and rates.