Reference-free video-to-real distance approximation-based urban social distancing analytics amid COVID-19 pandemic

Real-time social distance measurement and face mask detection in public transportation systems during the COVID-19 pandemic and post-pandemic Era: Theoretical approach and case study in Italy

Due to its remarkable learning ability and benefits in several areas of real-life, deep learning-based applications have recovered to be a research topic of great importance in the last few years. This article presents a method devoted to guaranteeing safety conditions in public transportation systems (PTS) during the COVID-19 pandemic and post-pandemic era. The paper describes a viable real-time model based on deep learning for monitoring social distance between users and detecting face masks in stop areas and inside vehicles of public transportation systems. Detections are made using the deep learning approach and YOLOv3 algorithm. The safety rule violations are represented by red bounding boxes and red circles in a bird's eye view as output of the video surveillance analysis. The datasets used to train the neural network are the "Caltech Pedestrian Dataset" and the "COVID-19 Medical Face Mask Detection Dataset". Metrics, such Loss Accuracy, and Precision, obtained in the testing process of the neural network were used to evaluate the performance of the model in detecting users and face masks. The proposed method was recently tested in the Public Transportation System of the Municipality of Piazza Armerina (Italy). The results show a significant reliability of the method in detecting real-time interactions between users of the PTS in terms of over-time variations in their mutual distancing, as well as in recognising cases of violation of the imposed social distancing and FFP2 face mask use.

Performance analysis of U-Net with hybrid loss for foreground detection

With the latest developments in deep neural networks, the convolutional neural network (CNN) has made considerable progress in the area of foreground detection. However, the top-rank background subtraction algorithms for foreground detection still have many shortcomings. It is challenging to extract the true foreground against complex background. To tackle the bottleneck, we propose a hybrid loss-assisted U-Net framework for foreground detection. A proposed deep learning model integrates transfer learning and hybrid loss for better feature representation and faster model convergence. The core idea is to incorporate reference background image and change detection mask in the learning network. Furthermore, we empirically investigate the potential of hybrid loss over single loss function. The advantages of two significant loss functions are combined to tackle the class imbalance problem in foreground detection. The proposed technique demonstrates its effectiveness on standard datasets and performs better than the top-rank methods in challenging environment. Moreover, experiments on unseen videos also confirm the efficacy of proposed method.

Deep visual social distancing monitoring to combat COVID-19: A comprehensive survey

Since the start of the COVID-19 pandemic, social distancing (SD) has played an essential role in controlling and slowing down the spread of the virus in smart cities. To ensure the respect of SD in public areas, visual SD monitoring (VSDM) provides promising opportunities by (i) controlling and analyzing the physical distance between pedestrians in real-time, (ii) detecting SD violations among the crowds, and (iii) tracking and reporting individuals violating SD norms. To the authors' best knowledge, this paper proposes the first comprehensive survey of VSDM frameworks and identifies their challenges and future perspectives. Typically, we review existing contributions by presenting the background of VSDM, describing evaluation metrics, and discussing SD datasets. Then, VSDM techniques are carefully reviewed after dividing them into two main categories: hand-crafted feature-based and deep-learning-based methods. A significant focus is paid to convolutional neural networks (CNN)-based methodologies as most of the frameworks have used either one-stage, two-stage, or multi-stage CNN models. A comparative study is also conducted to identify their pros and cons. Thereafter, a critical analysis is performed to highlight the issues and impediments that hold back the expansion of VSDM systems. Finally, future directions attracting significant research and development are derived.

Crowd dynamics research in the era of Covid-19 pandemic: Challenges and opportunities

With the issues of crowd control and physical distancing becoming central to disease prevention measures, one would expect that crowd research should become a focus of attention during the Covid-19 pandemic era. However, I will show, based on a variety of metrics, that not only has this not been the case, but also, the first two years of the pandemic have posed an undisputable setback to the development and growth of crowd science. Without intervention, this could potentially aggravate further and cause a long-lasting recession in this field. This article, in addition to documenting and highlighting this issue, aims to outline potential avenues through which crowd research can reshape itself in the era of Covid-19 pandemic, maintain its pre-pandemic momentum and even further expand the diversity of its topics. Despite significant changes that the pandemic has brought to human life, issues related to congregation and mobility of pedestrians, building fires, crowd incidents, rallying crowds and the like have not disappeared from societies and remain relevant. Moreover, the diversity of pandemic-related problems itself creates a rich ground for making novel scientific discoveries. This could provide grounds for establishing fresh dimensions in crowd dynamics research. These potential new dimensions extend to all areas of this field including numerical and experimental investigations, crowd psychology and applications of computer vision and artificial intelligence methods in crowd management. The Covid-19 pandemic may have posed challenges to crowd researchers but has also created ample potential opportunities. This is further evidenced by reviewing efforts taken thus far in pandemic-related crowd research.

Motion-compensated online object tracking for activity detection and crowd behavior analysis

It is a nontrivial task to manage crowds in public places and recognize unacceptable behavior (such as violating social distancing norms during the COVID-19 pandemic). In such situations, people should avoid loitering (unnecessary moving out in public places without apparent purpose) and maintain a sufficient physical distance. In this study, a multi-object tracking algorithm has been introduced to improve short-term object occlusion, detection errors, and identity switches. The objects are tracked through bounding box detection and with linear velocity estimation of the object using the Kalman filter frame by frame. The predicted tracks are kept alive for some time, handling the missing detections and short-term object occlusion. ID switches (mainly due to crossing trajectories) are managed by explicitly considering the motion direction of the objects in real time. Furthermore, a novel approach to detect unusual behavior of loitering with a severity level is proposed based on the tracking information. An adaptive algorithm is also proposed to detect physical distance violation based on the object dimensions for the entire length of the track. At last, a mathematical approach to calculate actual physical distance is proposed by using the height of a human as a reference object which adheres more specific distancing norms. The proposed approach is evaluated in traffic and pedestrian movement scenarios. The experimental results demonstrate a significant improvement in the results.

Monitoring social-distance in wide areas during pandemics: a density map and segmentation approach

With the relaxation of the containment measurements around the globe, monitoring the social distancing in crowded public spaces is of great importance to prevent a new massive wave of COVID-19 infections. Recent works in that matter have limited themselves by assessing social distancing in corridors up to small crowds by detecting each person individually, considering the full body in the image. In this work, we propose a new framework for monitoring the social-distance using end-to-end Deep Learning, to detect crowds violating social-distancing in wide areas, where important occlusions may be present. Our framework consists in the creation of new ground truth social distance labels, based on the ground truth density maps, and the proposal of two different solutions, a density-map-based and a segmentation-based, to detect crowds violating social-distancing constraints. We assess the results of both approaches by using the generated ground truth from the PET2009 and CityStreet datasets. We show that our framework performs well at providing the zones where people are not following the social-distance, even when heavily occluded or far away from the camera, compared to current detection and tracking approaches.

A Social Distance Estimation and Crowd Monitoring System for Surveillance Cameras

Social distancing is crucial to restrain the spread of diseases such as COVID-19, but complete adherence to safety guidelines is not guaranteed. Monitoring social distancing through mass surveillance is paramount to develop appropriate mitigation plans and exit strategies. Nevertheless, it is a labor-intensive task that is prone to human error and tainted with plausible breaches of privacy. This paper presents a privacy-preserving adaptive social distance estimation and crowd monitoring solution for camera surveillance systems. We develop a novel person localization strategy through pose estimation, build a privacy-preserving adaptive smoothing and tracking model to mitigate occlusions and noisy/missing measurements, compute inter-personal distances in the real-world coordinates, detect social distance infractions, and identify overcrowded regions in a scene. Performance evaluation is carried out by testing the system's ability in person detection, localization, density estimation, anomaly recognition, and high-risk areas identification. We compare the proposed system to the latest techniques and examine the performance gain delivered by the localization and smoothing/tracking algorithms. Experimental results indicate a considerable improvement, across different metrics, when utilizing the developed system. In addition, they show its potential and functionality for applications other than social distancing.

Street life and pedestrian activities in smart cities: opportunities and challenges for computational urban science

Ongoing efforts among cities to reinvigorate streets have encouraged innovations in using smart data to understand pedestrian activities. Empowered by advanced algorithms and computation power, data from smartphone applications, GPS devices, video cameras, and other forms of sensors can help better understand and promote street life and pedestrian activities. Through adopting a pedestrian-oriented and place-based approach, this paper reviews the major environmental components, pedestrian behavior, and sources of smart data in advancing this field of computational urban science. Responding to the identified research gap, a case study that hybridizes different smart data to understand pedestrian jaywalking as a reflection of urban spaces that need further improvement is presented. Finally, some major research challenges and directions are also highlighted.