Interactive Visual Discovering of Movement Patterns from Sparsely Sampled Geo-tagged Social Media Data

A survey of urban visual analytics: Advances and future directions

Developing effective visual analytics systems demands care in characterization of domain problems and integration of visualization techniques and computational models. Urban visual analytics has already achieved remarkable success in tackling urban problems and providing fundamental services for smart cities. To promote further academic research and assist the development of industrial urban analytics systems, we comprehensively review urban visual analytics studies from four perspectives. In particular, we identify 8 urban domains and 22 types of popular visualization, analyze 7 types of computational method, and categorize existing systems into 4 types based on their integration of visualization techniques and computational models. We conclude with potential research directions and opportunities.

Data science for pedestrian and high street retailing as a framework for advancing urban informatics to individual scales

Background

In this paper, we consider the applicability of the customer journey framework from retailing as a driver for urban informatics at individual scales within urban science. The customer journey considers shopper experiences in the context of shopping paths, retail service spaces, and touch-points that draw them into contact. Around this framework, retailers have developed sophisticated data science for observation, identification, and measurement of customers in the context of their shopping behavior. This knowledge supports broad data-driven understanding of customer experiences in physical spaces, economic spaces of decision and choice, persuasive spaces of advertising and branding, and inter-personal spaces of customer-staff interaction.

Method

We review the literature on pedestrian and high street retailing, and on urban informatics. We investigate whether the customer journey could be usefully repurposed for urban applications. Specifically, we explore the potential use of the customer journey framework for producing new insight into pedestrian behavior, where a sort of empirical hyperopia has long abounded because data are always in short supply.

Results

Our review addresses how the customer journey might be used as a structure for examining how urban walkers come into contact with the built environment, how people actively and passively sense and perceive ambient city life as they move, how pedestrians make sense of urban context, and how they use this knowledge to build cognition of city streetscapes. Each of these topics has relevance to walking studies specifically, but also to urban science more generally. We consider how retailing might reciprocally benefit from urban science perspectives, especially in extending the reach of retailers' insight beyond store walls, into the retail high streets from which they draw custom.

Conclusion

We conclude that a broad set of theoretical frameworks, data collection schemes, and analytical methodologies that have advanced retail data science closer and closer to individual-level acumen might be usefully applied to accomplish the same in urban informatics. However, we caution that differences between retailers' and urban scientists' viewpoints on privacy presents potential controversy.

Mobility Inference on Long-Tailed Sparse Trajectory

Analyzing the urban trajectory in cities has become an important topic in data mining. How can we model the human mobility consisting of stay and travel states from the raw trajectory data? How can we infer these mobility states from a single user’s trajectory information? How can we further generalize the mobility inference to the real-world trajectory data that span multiple users and are sparsely sampled over time?

In this paper, based on formal and rigid definitions of the stay/travel mobility, we propose a single trajectory inference algorithm that utilizes a generic long-tailed sparsity pattern in the large-scale trajectory data. The algorithm guarantees a 100% precision in the stay/travel inference with a provable lower bound in the recall metric. Furthermore, we design a transformer-like deep learning architecture on the problem of mobility inference from multiple sparse trajectories. Several adaptations from the standard transformer network structure are introduced, including the singleton design to avoid the negative effect of sparse labels in the decoder side, the customized space-time embedding on features of location records, and the mask apparatus at the output side for loss function correction. Evaluations on three trajectory datasets of 40 million urban users validate the performance guarantees of the proposed inference algorithm and demonstrate the superiority of our deep learning model, in comparison to sequence learning methods in the literature. On extremely sparse trajectories, the deep learning model improves from the single trajectory inference algorithm with more than two times of overall and F1 accuracy. The model also generalizes to large-scale trajectory data from different sources with good scalability.

Do You Believe Your (Social Media) Data? A Personal Story on Location Data Biases, Errors, and Plausibility as Well as Their Visualization

We present a case study on a journey about a personal data collection of carnivorous plant species habitats, and the resulting scientific exploration of location data biases, data errors, location hiding, and data plausibility. While initially driven by personal interest, our work led to the analysis and development of various means for visualizing threats to insight from geo-tagged social media data. In the course of this endeavor we analyzed local and global geographic distributions and their inaccuracies. We also contribute Motion Plausibility Profiles-a new means for visualizing how believable a specific contributor's location data is or if it was likely manipulated. We then compared our own repurposed social media dataset with data from a dedicated citizen science project. Compared to biases and errors in the literature on traditional citizen science data, with our visualizations we could also identify some new types or show new aspects for known ones. Moreover, we demonstrate several types of errors and biases for repurposed social media data. Please note that people with color impairments may consider our alternative paper version.

Social Network Analysis: A Survey on Measure, Structure, Language Information Analysis, Privacy, and Applications

The rapid growth in popularity of online social networks provides new opportunities in computer science, sociology, math, information studies, biology, business, and more. Social network analysis (SNA) is a paramount technique supporting understanding social relationships and networks. Accordingly, certain studies and reviews have been presented focusing on information dissemination, influence analysis, link prediction, and more. However, the ultimate aim is for social network background knowledge and analysis to solve real-world social network problems. SNA still has several research challenges in this context, including users’ privacy in online social networks. Inspired by these facts, we have presented a survey on social network analysis techniques, visualization, structure, privacy, and applications. This detailed study has started with the basics of network representation, structure, and measures. Our primary focus is on SNA applications with state-of-the-art techniques. We further provide a comparative analysis of recent developments on SNA problems in the sequel. The privacy preservation with SNA is also surveyed. In the end, research challenges and future directions are discussed to suggest researchers a starting point for their research.

Visual Cascade Analytics of Large-Scale Spatiotemporal Data

Many spatiotemporal events can be viewed as contagions. These events implicitly propagate across space and time by following cascading patterns, expanding their influence, and generating event cascades that involve multiple locations. Analyzing such cascading processes presents valuable implications in various urban applications, such as traffic planning and pollution diagnostics. Motivated by the limited capability of the existing approaches in mining and interpreting cascading patterns, we propose a visual analytics system called VisCas. VisCas combines an inference model with interactive visualizations and empowers analysts to infer and interpret the latent cascading patterns in the spatiotemporal context. To develop VisCas, we address three major challenges 1) generalized pattern inference; 2) implicit influence visualization; and 3) multifaceted cascade analysis. For the first challenge, we adapt the state-of-the-art cascading network inference technique to general urban scenarios, where cascading patterns can be reliably inferred from large-scale spatiotemporal data. For the second and third challenges, we assemble a set of effective visualizations to support location navigation, influence inspection, and cascading exploration, and facilitate the in-depth cascade analysis. We design a novel influence view based on a three-fold optimization strategy for analyzing the implicit influences of the inferred patterns. We demonstrate the capability and effectiveness of VisCas with two case studies conducted on real-world traffic congestion and air pollution datasets with domain experts.

Compass: Towards Better Causal Analysis of Urban Time Series

The spatial time series generated by city sensors allow us to observe urban phenomena like environmental pollution and traffic congestion at an unprecedented scale. However, recovering causal relations from these observations to explain the sources of urban phenomena remains a challenging task because these causal relations tend to be time-varying and demand proper time series partitioning for effective analyses. The prior approaches extract one causal graph given long-time observations, which cannot be directly applied to capturing, interpreting, and validating dynamic urban causality. This paper presents Compass, a novel visual analytics approach for in-depth analyses of the dynamic causality in urban time series. To develop Compass, we identify and address three challenges: detecting urban causality, interpreting dynamic causal relations, and unveiling suspicious causal relations. First, multiple causal graphs over time among urban time series are obtained with a causal detection framework extended from the Granger causality test. Then, a dynamic causal graph visualization is designed to reveal the time-varying causal relations across these causal graphs and facilitate the exploration of the graphs along the time. Finally, a tailored multi-dimensional visualization is developed to support the identification of spurious causal relations, thereby improving the reliability of causal analyses. The effectiveness of Compass is evaluated with two case studies conducted on the real-world urban datasets, including the air pollution and traffic speed datasets, and positive feedback was received from domain experts.

Visual Evaluation for Autonomous Driving

Autonomous driving technologies often use state-of-the-art artificial intelligence algorithms to understand the relationship between the vehicle and the external environment, to predict the changes of the environment, and then to plan and control the behaviors of the vehicle accordingly. The complexity of such technologies makes it challenging to evaluate the performance of autonomous driving systems and to find ways to improve them. The current approaches to evaluating such autonomous driving systems largely use a single score to indicate the overall performance of a system, but domain experts have difficulties in understanding how individual components or algorithms in an autonomous driving system may contribute to the score. To address this problem, we collaborate with domain experts on autonomous driving algorithms, and propose a visual evaluation method for autonomous driving. Our method considers the data generated in all components during the whole process of autonomous driving, including perception results, planning routes, prediction of obstacles, various controlling parameters, and evaluation of comfort. We develop a visual analytics workflow to integrate an evaluation mathematical model with adjustable parameters, support the evaluation of the system from the level of the overall performance to the level of detailed measures of individual components, and to show both evaluation scores and their contributing factors. Our implemented visual analytics system provides an overview evaluation score at the beginning and shows the animation of the dynamic change of the scores at each period. Experts can interactively explore the specific component at different time periods and identify related factors. With our method, domain experts not only learn about the performance of an autonomous driving system, but also identify and access the problematic parts of each component. Our visual evaluation system can be applied to the autonomous driving simulation system and used for various evaluation cases. The results of using our system in some simulation cases and the feedback from involved domain experts confirm the usefulness and efficiency of our method in helping people gain in-depth insight into autonomous driving systems.

UrbanMotion: Visual Analysis of Metropolitan-Scale Sparse Trajectories

Visualizing massive scale human movement in cities plays an important role in solving many of the problems that modern cities face (e.g., traffic optimization, business site configuration). In this article, we study a big mobile location dataset that covers millions of city residents, but is temporally sparse on the trajectory of individual user. Mapping sparse trajectories to illustrate population movement poses several challenges from both analysis and visualization perspectives. In the literature, there are a few techniques designed for sparse trajectory visualization; yet they do not consider trajectories collected from mobile apps that possess long-tailed sparsity with record intervals as long as hours. This article introduces UrbanMotion, a visual analytics system that extends the original wind map design by supporting map-matched local movements, multi-directional population flows, and population distributions. Effective methods are proposed to extract and aggregate population movements from dense parts of the trajectories leveraging their long-tailed sparsity. Both characteristic and anomalous patterns are discovered and visualized. We conducted three case studies, one comparative experiment, and collected expert feedback in the application domains of commuting analysis, event detection, and business site configuration. The study result demonstrates the significance and effectiveness of our system in helping to complete key analytics tasks for urban users.

Semantic Enhancement of Human Urban Activity Chain Construction Using Mobile Phone Signaling Data

Data-driven urban human activity mining has become a hot topic of urban dynamic modeling and analysis. Semantic activity chain modeling with activity purpose provides scientific methodological support for the analysis and decision-making of human behavior, urban planning, traffic management, green sustainable development, etc. However, the spatial and temporal uncertainty of the ubiquitous mobile sensing data brings a huge challenge for modeling and analyzing human activities. Existing approaches for modeling and identifying human activities based on massive social sensing data rely on a large number of valid supervised samples or limited prior knowledge. This paper proposes an effective methodology for building human activity chains based on mobile phone signaling data and labeling activity purpose semantics to analyze human activity patterns, spatiotemporal behavior, and urban dynamics. We fully verified the effectiveness and accuracy of the proposed method in human daily activity process construction and activity purpose identification through accuracy comparison and spatial-temporal distribution exploration. This study further confirms the possibility of using big data to observe urban human spatiotemporal behavior.

GLoG: Laplacian of Gaussian for Spatial Pattern Detection in Spatio-Temporal Data

Boundary detection has long been a fundamental tool for image processing and computer vision, supporting the analysis of static and time-varying data. In this work, we built upon the theory of Graph Signal Processing to propose a novel boundary detection filter in the context of graphs, having as main application scenario the visual analysis of spatio-temporal data. More specifically, we propose the equivalent for graphs of the so-called Laplacian of Gaussian edge detection filter, which is widely used in image processing. The proposed filter is able to reveal interesting spatial patterns while still enabling the definition of entropy of time slices. The entropy reveals the degree of randomness of a time slice, helping users to identify expected and unexpected phenomena over time. The effectiveness of our approach appears in applications involving synthetic and real data sets, which show that the proposed methodology is able to uncover interesting spatial and temporal phenomena. The provided examples and case studies make clear the usefulness of our approach as a mechanism to support visual analytic tasks involving spatio-temporal data.

A Natural-language-based Visual Query Approach of Uncertain Human Trajectories

Visual querying is essential for interactively exploring massive trajectory data. However, the data uncertainty imposes profound challenges to fulfill advanced analytics requirements. On the one hand, many underlying data does not contain accurate geographic coordinates, e.g., positions of a mobile phone only refer to the regions (i.e., mobile cell stations) in which it resides, instead of accurate GPS coordinates. On the other hand, domain experts and general users prefer a natural way, such as using a natural language sentence, to access and analyze massive movement data. In this paper, we propose a visual analytics approach that can extract spatial-temporal constraints from a textual sentence and support an effective query method over uncertain mobile trajectory data. It is built up on encoding massive, spatially uncertain trajectories by the semantic information of the POls and regions covered by them, and then storing the trajectory documents in text database with an effective indexing scheme. The visual interface facilitates query condition specification, situation-aware visualization, and semantic exploration of large trajectory data. Usage scenarios on real-world human mobility datasets demonstrate the effectiveness of our approach.

AirVis: Visual Analytics of Air Pollution Propagation

Air pollution has become a serious public health problem for many cities around the world. To find the causes of air pollution, the propagation processes of air pollutants must be studied at a large spatial scale. However, the complex and dynamic wind fields lead to highly uncertain pollutant transportation. The state-of-the-art data mining approaches cannot fully support the extensive analysis of such uncertain spatiotemporal propagation processes across multiple districts without the integration of domain knowledge. The limitation of these automated approaches motivates us to design and develop AirVis, a novel visual analytics system that assists domain experts in efficiently capturing and interpreting the uncertain propagation patterns of air pollution based on graph visualizations. Designing such a system poses three challenges: a) the extraction of propagation patterns; b) the scalability of pattern presentations; and c) the analysis of propagation processes. To address these challenges, we develop a novel pattern mining framework to model pollutant transportation and extract frequent propagation patterns efficiently from large-scale atmospheric data. Furthermore, we organize the extracted patterns hierarchically based on the minimum description length (MDL) principle and empower expert users to explore and analyze these patterns effectively on the basis of pattern topologies. We demonstrated the effectiveness of our approach through two case studies conducted with a real-world dataset and positive feedback from domain experts.

COPE: Interactive Exploration of Co-Occurrence Patterns in Spatial Time Series

Spatial time series is a common type of data dealt with in many domains, such as economic statistics and environmental science. There have been many studies focusing on finding and analyzing various kinds of events in time series; the term 'event' refers to significant changes or occurrences of particular patterns formed by consecutive attribute values. We focus on a further step in event analysis: discover temporal relationship patterns between event locations, i.e., repeated cases when there is a specific temporal relationship (same time, before, or after) between events occurring at two locations. This can provide important clues for understanding the formation and spreading mechanisms of events and interdependencies among spatial locations. We propose a visual exploration framework COPE (Co-Occurrence Pattern Exploration), which allows users to extract events of interest from data and detect various co-occurrence patterns among them. Case studies and expert reviews were conducted to verify the effectiveness and scalability of COPE using two real-world datasets.

Diverse Visualization Techniques and Methods of Moving-Object-Trajectory Data: A Review

Trajectory big data have significant applications in many areas, such as traffic management, urban planning and military reconnaissance. Traditional visualization methods, which are represented by contour maps, shading maps and hypsometric maps, are mainly based on the spatiotemporal information of trajectories, which can macroscopically study the spatiotemporal conditions of the entire trajectory set and microscopically analyze the individual movement of each trajectory; such methods are widely used in screen display and flat mapping. With the improvement of trajectory data quality, these data can generally describe information in the spatial and temporal dimensions and involve many other attributes (e.g., speed, orientation, and elevation) with large data amounts and high dimensions. Additionally, these data have relatively complicated internal relationships and regularities, whose analysis could cause many troubles; the traditional approaches can no longer fully meet the requirements of visualizing trajectory data and mining hidden information. Therefore, diverse visualization methods that present the value of massive trajectory information are currently a hot research topic. This paper summarizes the research status of trajectory data-visualization techniques in recent years and extracts common contemporary trajectory data-visualization methods to comprehensively cognize and understand the fundamental characteristics and diverse achievements of trajectory-data visualization.

D-Map+

Information diffusion analysis is important in social media. In this work, we present a coherent ego-centric and event-centric model to investigate diffusion patterns and user behaviors. Applying the model, we propose Diffusion Map+ (D-Maps+), a novel visualization method to support exploration and analysis of user behaviors and diffusion patterns through a map metaphor. For ego-centric analysis, users who participated in reposting (i.e., resending a message initially posted by others) one central user’s posts (i.e., a series of original tweets) are collected. Event-centric analysis focuses on multiple central users discussing a specific event, with all the people participating and reposting messages about it. Social media users are mapped to a hexagonal grid based on their behavior similarities and in the chronological order of repostings. With the additional interactions and linkings, D-Map+ is capable of providing visual profiling of influential users, describing their social behaviors and analyzing the evolution of significant events in social media. A comprehensive visual analysis system is developed to support interactive exploration with D-Map+. We evaluate our work with real-world social media data and find interesting patterns among users and events. We also perform evaluations including user studies and expert feedback to certify the capabilities of our method.

Visualization of Urban Mobility Data from Intelligent Transportation Systems

Intelligent Transportation Systems are an important enabler for the smart cities paradigm. Currently, such systems generate massive amounts of granular data that can be analyzed to better understand people’s dynamics. To address the multivariate nature of spatiotemporal urban mobility data, researchers and practitioners have developed an extensive body of research and interactive visualization tools. Data visualization provides multiple perspectives on data and supports the analytical tasks of domain experts. This article surveys related studies to analyze which topics of urban mobility were addressed and their related phenomena, and to identify the adopted visualization techniques and sensors data types. We highlight research opportunities based on our findings.

A Visual Analytics Framework for Exploring Theme Park Dynamics

In 2015, the top 10 largest amusement park corporations saw a combined annual attendance of over 400 million visitors. Daily average attendance in some of the most popular theme parks in the world can average 44,000 visitors per day. These visitors ride attractions, shop for souvenirs, and dine at local establishments; however, a critical component of their visit is the overall park experience. This experience depends on the wait time for rides, the crowd flow in the park, and various other factors linked to the crowd dynamics and human behavior. As such, better insight into visitor behavior can help theme parks devise competitive strategies for improved customer experience. Research into the use of attractions, facilities, and exhibits can be studied, and as behavior profiles emerge, park operators can also identify anomalous behaviors of visitors which can improve safety and operations. In this article, we present a visual analytics framework for analyzing crowd dynamics in theme parks. Our proposed framework is designed to support behavioral analysis by summarizing patterns and detecting anomalies. We provide methodologies to link visitor movement data, communication data, and park infrastructure data. This combination of data sources enables a semantic analysis of who , what , when , and where , enabling analysts to explore visitor-visitor interactions and visitor-infrastructure interactions. Analysts can identify behaviors at the macro level through semantic trajectory clustering views for group behavior dynamics, as well as at the micro level using trajectory traces and a novel visitor network analysis view. We demonstrate the efficacy of our framework through two case studies of simulated theme park visitors.

Cognitive Visualization of Popular Regions Discovered From Geo-Tagged Social Media Data

This article focuses on the cognitive exploration of photo sharing data which contain information about the location where the photo was taken and potentially some description about the photo. Therefore, the features of photo-spots can be deduced. Spots with similar features constitute a region of cognitive interest. The objective is to identify these regions and allow users to explore into regions of interest by cognitive understanding of their features. The authors propose an approach that makes use of semantic analysis, data clustering, and cognitive visualization. In this article, the authors introduce the design of an interactive visualization interface which projects photo sharing data to cognitive social activity map components. The contributions are two-fold. First, the authors put forward a novel social-media data classification method. Second, the authors suggest a new method to explore social activity maps by discovering regions of cognitive interest. Experiments are performed on the Flickr dataset.

StreetVizor: Visual Exploration of Human-Scale Urban Forms Based on Street Views

Urban forms at human-scale, i.e., urban environments that individuals can sense (e.g., sight, smell, and touch) in their daily lives, can provide unprecedented insights on a variety of applications, such as urban planning and environment auditing. The analysis of urban forms can help planners develop high-quality urban spaces through evidence-based design. However, such analysis is complex because of the involvement of spatial, multi-scale (i.e., city, region, and street), and multivariate (e.g., greenery and sky ratios) natures of urban forms. In addition, current methods either lack quantitative measurements or are limited to a small area. The primary contribution of this work is the design of StreetVizor, an interactive visual analytics system that helps planners leverage their domain knowledge in exploring human-scale urban forms based on street view images. Our system presents two-stage visual exploration: 1) an AOI Explorer for the visual comparison of spatial distributions and quantitative measurements in two areas-of-interest (AOIs) at city- and region-scales; 2) and a Street Explorer with a novel parallel coordinate plot for the exploration of the fine-grained details of the urban forms at the street-scale. We integrate visualization techniques with machine learning models to facilitate the detection of street view patterns. We illustrate the applicability of our approach with case studies on the real-world datasets of four cities, i.e., Hong Kong, Singapore, Greater London and New York City. Interviews with domain experts demonstrate the effectiveness of our system in facilitating various analytical tasks.

Urban Pulse: Capturing the Rhythm of Cities

Cities are inherently dynamic. Interesting patterns of behavior typically manifest at several key areas of a city over multiple temporal resolutions. Studying these patterns can greatly help a variety of experts ranging from city planners and architects to human behavioral experts. Recent technological innovations have enabled the collection of enormous amounts of data that can help in these studies. However, techniques using these data sets typically focus on understanding the data in the context of the city, thus failing to capture the dynamic aspects of the city. The goal of this work is to instead understand the city in the context of multiple urban data sets. To do so, we define the concept of an "urban pulse" which captures the spatio-temporal activity in a city across multiple temporal resolutions. The prominent pulses in a city are obtained using the topology of the data sets, and are characterized as a set of beats. The beats are then used to analyze and compare different pulses. We also design a visual exploration framework that allows users to explore the pulses within and across multiple cities under different conditions. Finally, we present three case studies carried out by experts from two different domains that demonstrate the utility of our framework.