Do You Believe Your (Social Media) Data? A Personal Story on Location Data Biases, Errors, and Plausibility as Well as Their Visualization

We present a case study on a journey about a personal data collection of carnivorous plant species habitats, and the resulting scientific exploration of location data biases, data errors, location hiding, and data plausibility. While initially driven by personal interest, our work led to the analysis and development of various means for visualizing threats to insight from geo-tagged social media data. In the course of this endeavor we analyzed local and global geographic distributions and their inaccuracies. We also contribute Motion Plausibility Profiles-a new means for visualizing how believable a specific contributor's location data is or if it was likely manipulated. We then compared our own repurposed social media dataset with data from a dedicated citizen science project. Compared to biases and errors in the literature on traditional citizen science data, with our visualizations we could also identify some new types or show new aspects for known ones. Moreover, we demonstrate several types of errors and biases for repurposed social media data. Please note that people with color impairments may consider our alternative paper version.

A Comparative Study of Methods for the Visualization of Probability Distributions of Geographical Data

Probability distributions are omnipresent in data analysis. They are often used to model the natural uncertainty present in real phenomena or to describe the properties of a data set. Designing efficient visual metaphors to convey probability distributions is, however, a difficult problem. This fact is especially true for geographical data, where conveying the spatial context constrains the design space. While many different alternatives have been proposed to solve this problem, they focus on representing data variability. However, they are not designed to support spatial analytical tasks involving probability quantification. The present work aims to adapt recent non-spatial approaches to the geographical context, in order to support probability quantification tasks. We also present a user study that compares the efficiency of these approaches in terms of both accuracy and usability.

Uncertainty in geospatial health: challenges and opportunities ahead

PURPOSE

Uncertainty is not always well captured, understood, or modeled properly, and can bias the robustness of complex relationships, such as the association between the environment and public health through exposure, estimates of geographic accessibility and cluster detection, to name a few.

METHODS

We review current challenges and future opportunities as geospatial data and analyses are applied to the field of public health. We are particularly interested in the sources of uncertainty in geospatial data and how this uncertainty may propagate in spatial analysis.

RESULTS

We present opportunities to reduce the magnitude and impact of uncertainty. Specifically, we focus on (1) the use of multiple reference data sources to reduce geocoding errors, (2) the validity of online geocoders and how confidentiality (e.g., HIPAA) may be breached, (3) use of multiple reference data sources to reduce geocoding errors, (4) the impact of geoimputation techniques on travel estimates, (5) residential mobility and how it affects accessibility metrics and clustering, and (6) modeling errors in the American Community Survey. Our paper discusses how to communicate spatial and spatiotemporal uncertainty, and high-performance computing to conduct large amounts of simulations to ultimately increase statistical robustness for studies in public health.

CONCLUSIONS

Our paper contributes to recent efforts to fill in knowledge gaps at the intersection of spatial uncertainty and public health.

A comparison of the performance on extrinsic and intrinsic cartographic visualizations through correctness, response time and cognitive processing

The aim of this study was to compare the performance of two bivariate visualizations by measuring response correctness (error rate) and response time, and to identify the differences in cognitive processes involved in map-reading tasks by using eye-tracking methods. The present study is based on our previous research and the hypothesis that the use of different visualization methods may lead to significant cognitive-processing differences. We applied extrinsic and intrinsic visualizations in the study. Participants in the experiment were presented maps which depicted two variables (soil moisture and soil depth) and asked to identify the areas which displayed either a single condition (e.g., “find an area with low soil depth”) or both conditions (e.g., “find an area with high soil moisture and low soil depth”). The research sample was composed of 31 social sciences and humanities university students. The experiment was performed under laboratory conditions, and Hypothesis software was used for data collection. Eye-tracking data were collected for 23 of the participants. An SMI RED-m eye-tracker was used to determine whether either of the two visualization methods was more efficient for solving the given map-reading tasks. Our results showed that with the intrinsic visualization method, the participants spent significantly more time with the map legend. This result suggests that extrinsic and intrinsic visualizations induce different cognitive processes. The intrinsic method was observed to generally require more time and led to higher error rates. In summary, the extrinsic method was found to be more efficient than the intrinsic method, although the difference was less pronounced in the tasks which contained two variables, which proved to be better suited to intrinsic visualization.

Visualizing Uncertainty for Non-Expert End Users: The Challenge of the Deterministic Construal Error

There is a growing body of evidence that numerical uncertainty expressions can be used by non-experts to improve decision quality. Moreover, there is some evidence that similar advantages extend to graphic expressions of uncertainty. However, visualizing uncertainty introduces challenges as well. Here, we discuss key misunderstandings that may arise from uncertainty visualizations, in particular the evidence that users sometimes fail to realize that the graphic depicts uncertainty. Instead they have a tendency to interpret the image as representing some deterministic quantity. We refer to this as the deterministic construal error. Although there is now growing evidence for the deterministic construal error, few studies are designed to detect it directly because they inform participants upfront that the visualization expresses uncertainty. In a natural setting such cues would be absent, perhaps making the deterministic assumption more likely. Here we discuss the psychological roots of this key but underappreciated misunderstanding as well as possible solutions. This is a critical question because it is now clear that members of the public understand that predictions involve uncertainty and have greater trust when uncertainty is included. Moreover, they can understand and use uncertainty predictions to tailor decisions to their own risk tolerance, as long as they are carefully expressed, taking into account the cognitive processes involved.

Achieving Accuracy through Ambiguity: the Interactivity of Risk Communication in Severe Weather Events

Risks associated with natural hazards such as hurricanes are increasingly communicated on social media. For hurricane risk communication, visual information products—graphics—generated by meteorologists and scientists at weather agencies portray forecasts and atmospheric conditions and are offered to parsimoniously convey predictions of severe storms. This research considers risk interactivity by examining a particular hurricane graphic which has shown in previous research to have a distinctive diffusion signature: the ‘spaghetti plot’, which contains multiple discrete lines depicting a storm’s possible path. We first analyzed a large dataset of microblog interactions around spaghetti plots between members of the public and authoritative weather sources within the US during the 2017 Atlantic hurricane season. We then conducted interviews with a sample of the weather authorities after preliminary findings sketched the role that experts have in such communications. Findings describe how people make sense of risk dialogically over graphics, and show the presence of a fundamental tension in risk communication between accuracy and ambiguity. The interactive effort combats the unintended declarative quality of the graphical risk representation through communicative acts that maintain a hazard’s inherent ambiguity until risk can be foreclosed. We consider theoretical and practice-based implications of the limits and potentials of graphical risk representations and of widely diffused scientific communication, and offer reasons we need CSCW attention paid to the larger enterprise of risk communication.

Exploring the Sensitivity of Choropleths under Attribute Uncertainty

The choropleth map is an essential tool for spatial data analysis. However, the underlying attribute values of a spatial unit greatly influence the statistical analyses and map classification procedures when generating a choropleth map. If the attribute values incorporate a range of uncertainty, a critical task is determining how much the uncertainty impacts both the map visualization and the statistical analysis. In this paper, we present a visual analytics system that enhances our understanding of the impact of attribute uncertainty on data visualization and statistical analyses of these data. Our system consists of a parallel coordinates-based uncertainty specification view, an impact river and impact matrix visualization for region-based and simulation-based analysis, and a dual-choropleth map and t-SNE plot for visualizing the changes in classification and spatial autocorrelation over the range of uncertainty in the attribute values. We demonstrate our system through three use cases illustrating the impact of attribute uncertainty in geographic analysis.

Accounting for Training Data Error in Machine Learning Applied to Earth Observations

Remote sensing, or Earth Observation (EO), is increasingly used to understand Earth system dynamics and create continuous and categorical maps of biophysical properties and land cover, especially based on recent advances in machine learning (ML). ML models typically require large, spatially explicit training datasets to make accurate predictions. Training data (TD) are typically generated by digitizing polygons on high spatial-resolution imagery, by collecting in situ data, or by using pre-existing datasets. TD are often assumed to accurately represent the truth, but in practice almost always have error, stemming from (1) sample design, and (2) sample collection errors. The latter is particularly relevant for image-interpreted TD, an increasingly commonly used method due to its practicality and the increasing training sample size requirements of modern ML algorithms. TD errors can cause substantial errors in the maps created using ML algorithms, which may impact map use and interpretation. Despite these potential errors and their real-world consequences for map-based decisions, TD error is often not accounted for or reported in EO research. Here we review the current practices for collecting and handling TD. We identify the sources of TD error, and illustrate their impacts using several case studies representing different EO applications (infrastructure mapping, global surface flux estimates, and agricultural monitoring), and provide guidelines for minimizing and accounting for TD errors. To harmonize terminology, we distinguish TD from three other classes of data that should be used to create and assess ML models: training reference data, used to assess the quality of TD during data generation; validation data, used to iteratively improve models; and map reference data, used only for final accuracy assessment. We focus primarily on TD, but our advice is generally applicable to all four classes, and we ground our review in established best practices for map accuracy assessment literature. EO researchers should start by determining the tolerable levels of map error and appropriate error metrics. Next, TD error should be minimized during sample design by choosing a representative spatio-temporal collection strategy, by using spatially and temporally relevant imagery and ancillary data sources during TD creation, and by selecting a set of legend definitions supported by the data. Furthermore, TD error can be minimized during the collection of individual samples by using consensus-based collection strategies, by directly comparing interpreted training observations against expert-generated training reference data to derive TD error metrics, and by providing image interpreters with thorough application-specific training. We strongly advise that TD error is incorporated in model outputs, either directly in bias and variance estimates or, at a minimum, by documenting the sources and implications of error. TD should be fully documented and made available via an open TD repository, allowing others to replicate and assess its use. To guide researchers in this process, we propose three tiers of TD error accounting standards. Finally, we advise researchers to clearly communicate the magnitude and impacts of TD error on map outputs, with specific consideration given to the likely map audience.

Why Authors Don't Visualize Uncertainty

Clear presentation of uncertainty is an exception rather than rule in media articles, data-driven reports, and consumer applications, despite proposed techniques for communicating sources of uncertainty in data. This work considers, Why do so many visualization authors choose not to visualize uncertainty? I contribute a detailed characterization of practices, associations, and attitudes related to uncertainty communication among visualization authors, derived from the results of surveying 90 authors who regularly create visualizations for others as part of their work, and interviewing thirteen influential visualization designers. My results highlight challenges that authors face and expose assumptions and inconsistencies in beliefs about the role of uncertainty in visualization. In particular, a clear contradiction arises between authors' acknowledgment of the value of depicting uncertainty and the norm of omitting direct depiction of uncertainty. To help explain this contradiction, I present a rhetorical model of uncertainty omission in visualization-based communication. I also adapt a formal statistical model of how viewers judge the strength of a signal in a visualization to visualization-based communication, to argue that uncertainty communication necessarily reduces degrees of freedom in viewers' statistical inferences. I conclude with recommendations for how visualization research on uncertainty communication could better serve practitioners' current needs and values while deepening understanding of assumptions that reinforce uncertainty omission.

Visualisation of Spatial Data Uncertainty. A Case Study of a Database of Topographic Objects

The Database of Topographic Objects (DTO) is the official database of Poland for collecting and providing spatial data with the detail level of a topographic map. Polish national DTOs manage information about the spatial location and attribute values of geographic objects. Data in the DTO are the starting point for geographic information systems (GISs) for various central and local governments as well as private institutions. Every set of spatial data based on measurement-derived data is susceptible to uncertainty. Therefore, the widespread awareness of data uncertainty is of vital importance to all GIS users. Cartographic visualisation techniques are an effective approach to informing spatial dataset users about the uncertainty of the data. The objective of the research was to define a set of methods for visualising the DTO data uncertainty using expert know-how and experience. This set contains visualisation techniques for presenting three types of uncertainty: positional, attribute, and temporal. The positional uncertainty for point objects was presented using visual variables, object fill with hue colour and lightness, and glyphs placed at map symbol positions. The positional uncertainty for linear objects was presented using linear object contours made of dotted lines and glyphs at vertices. Fill grain density and contour crispness were employed to represent the positional uncertainty for surface objects. The attribute value uncertainty and the temporal uncertainty were represented using fill grain density and fill colour value. The proposed set of the DTO uncertainty visualisation methods provides a finite array of visualisation techniques that can be tested and juxtaposed. The visualisation methods were comprehensively evaluated in a survey among experts who use spatial databases. Results of user preference analysis have demonstrated that the set of the DTO data uncertainty visualisation techniques may be applied to the full extent. The future implementation of the proposed visualisation methods in GIS databases will help data users interpret values correctly.

Towards an Uncertainty-Aware Visualization in the Digital Humanities

As visualization becomes widespread in a broad range of cross-disciplinary academic domains, such as the digital humanities (DH), critical voices have been raised on the perils of neglecting the uncertain character of data in the visualization design process. Visualizations that, purposely or not, obscure or remove uncertainty in its different forms from the scholars’ vision may negatively affect the manner in which humanities scholars regard computational methods as useful tools in their daily work. In this paper, we address the issue of uncertainty representation in the context of the humanities from a theoretical perspective, in an attempt to provide the foundations of a framework that allows for the construction of ecological interface designs which are able to expose the computational power of the algorithms at play while, at the same time, respecting the particularities and needs of humanistic research. To this end, we review past uncertainty taxonomies in other domains typically related to the humanities and visualization, such as cartography and GIScience. From this review, we select an uncertainty taxonomy related to the humanities that we link to recent research in visualization for the DH. Finally, we bring a novel analytics method developed by other authors (Progressive Visual Analytics) into question, which we argue can be a good candidate to resolve the aforementioned difficulties in DH practice.

Visualizations Out of Context: Addressing Pitfalls of Real-Time Realistic Hazard Visualizations

Realistic 3D hazard visualizations based on advanced Geographic Information Systems (GIS) may be directly driven by hydrodynamic and wind model outputs (e.g., ADCIRC, the ADvanced CIRCulation Model) and hazard impact modeling (e.g., predicting damage to structures and infrastructure). These methods create new possibilities for representing hazard impacts and support the development of near-real-time hazard forecasting and communication tools. This paper considers the wider implications of using these storm visualizations in light of current frameworks in the context of landscape and urban planning and cartography that have addressed the use of realistic 3D visualizations. Visualizations used outside of engagement processes organized by experts risk misleading the public and may have consequences in terms of feelings of individual self-efficacy or perception of scientists behind the visualizations. In addition to summarizing the implications of using these visualizations outside of recommended practices, a research agenda is proposed to guide the development of real-time realistic and semi-realistic visualizations for future use in hazard communication. Development of a clearer use-case for real-time visualization capabilities is an essential first step if such work is to continue.

In Pursuit of Error: A Survey of Uncertainty Visualization Evaluation

Understanding and accounting for uncertainty is critical to effectively reasoning about visualized data. However, evaluating the impact of an uncertainty visualization is complex due to the difficulties that people have interpreting uncertainty and the challenge of defining correct behavior with uncertainty information. Currently, evaluators of uncertainty visualization must rely on general purpose visualization evaluation frameworks which can be ill-equipped to provide guidance with the unique difficulties of assessing judgments under uncertainty. To help evaluators navigate these complexities, we present a taxonomy for characterizing decisions made in designing an evaluation of an uncertainty visualization. Our taxonomy differentiates six levels of decisions that comprise an uncertainty visualization evaluation: the behavioral targets of the study, expected effects from an uncertainty visualization, evaluation goals, measures, elicitation techniques, and analysis approaches. Applying our taxonomy to 86 user studies of uncertainty visualizations, we find that existing evaluation practice, particularly in visualization research, focuses on Performance and Satisfaction-based measures that assume more predictable and statistically-driven judgment behavior than is suggested by research on human judgment and decision making. We reflect on common themes in evaluation practice concerning the interpretation and semantics of uncertainty, the use of confidence reporting, and a bias toward evaluating performance as accuracy rather than decision quality. We conclude with a concrete set of recommendations for evaluators designed to reduce the mismatch between the conceptualization of uncertainty in visualization versus other fields.

Effect of Visualization Training on Uncertain Spatial Trajectory Predictions

OBJECTIVE

The goal of this study was to explore the ways in which visualizations influence the prediction of uncertain spatial trajectories (e.g., the unknown path of a downed aircraft or future path of a hurricane) and participant overconfidence in such prediction.

BACKGROUND

Previous research indicated that spatial predictions of uncertain trajectories are challenging and are often associated with overconfidence. Introducing a visualization aid during training may improve the understanding of uncertainty and reduce overconfidence.

METHOD

Two experiments asked participants to predict the location of various trajectories at a future time. Mean and variance estimates were compared for participants who were provided with a visualization and those who were not.

RESULTS

In Experiment 1, participants exhibited less error in mean estimations when a linear visualization was present but performed worse than controls once the visualization was removed. Similar results were shown in Experiment 2, with a nonlinear visualization. However, in both experiments, participants who were provided with a visualization did not retain any advantage in their variance estimations once the visualization was removed.

CONCLUSIONS

Visualizations may support spatial predictions under uncertainty, but they are associated with benefits and costs for the underlying knowledge being developed.

APPLICATION

Visualizations have the potential to influence how people make spatial predictions in the presence of uncertainty. Properly designed and implemented visualizations may help mitigate the cognitive biases related to such predictions.

Geovisualizing attribute uncertainty of interval and ratio variables: a framework and an implementation for vector data

Geovisualization of attribute uncertainty helps users to recognize underlying processes of spatial data. However, it still lacks an availability of uncertainty visualization tools in a standard GIS environment. This paper proposes a framework for attribute uncertainty visualization by extending bivariate mapping techniques. Specifically, this framework utilizes two cartographic techniques, choropleth mapping and proportional symbol mapping based on the types of attributes. This framework is implemented as an extension of ArcGIS in which three types of visualization tools are available: overlaid symbols on a choropleth map, coloring properties to a proportional symbol map, and composite symbols.

User perception and interpretation of tornado probabilistic hazard information: Comparison of four graphical designs

Effective design for presenting severe weather information is important to reduce devastating consequences of severe weather. The Probabilistic Hazard Information (PHI) system for severe weather is being developed by NOAA National Severe Storms Laboratory (NSSL) to communicate probabilistic hazardous weather information. This study investigates the effects of four PHI graphical designs for tornado threat, namely, "four-color"," red-scale", "grayscale" and "contour", on users' perception, interpretation, and reaction to threat information. PHI is presented on either a map background or a radar background. Analysis showed that the accuracy was significantly higher and response time faster when PHI was displayed on map background as compared to radar background due to better contrast. When displayed on a radar background, "grayscale" design resulted in a higher accuracy of responses. Possibly due to familiarity, participants reported four-color design as their favorite design, which also resulted in the fastest recognition of probability levels on both backgrounds. Our study shows the importance of using intuitive color-coding and sufficient contrast in conveying probabilistic threat information via graphical design. We also found that users follows a rational perceiving-judging-feeling-and acting approach in processing probabilistic hazard information for tornado.

Spatial Variation in the Quality of American Community Survey Estimates

Social science research, public and private sector decisions, and allocations of federal resources often rely on data from the American Community Survey (ACS). However, this critical data source has high uncertainty in some of its most frequently used estimates. Using 2006-2010 ACS median household income estimates at the census tract scale as a test case, we explore spatial and nonspatial patterns in ACS estimate quality. We find that spatial patterns of uncertainty in the northern United States differ from those in the southern United States, and they are also different in suburbs than in urban cores. In both cases, uncertainty is lower in the former than the latter. In addition, uncertainty is higher in areas with lower incomes. We use a series of multivariate spatial regression models to describe the patterns of association between uncertainty in estimates and economic, demographic, and geographic factors, controlling for the number of responses. We find that these demographic and geographic patterns in estimate quality persist even after we account for the number of responses. Our results indicate that data quality varies across places, making cross-sectional analysis both within and across regions less reliable. Finally, we present advice for data users and potential solutions to the challenges identified.

The Role of Uncertainty, Awareness, and Trust in Visual Analytics

Visual analytics supports humans in generating knowledge from large and often complex datasets. Evidence is collected, collated and cross-linked with our existing knowledge. In the process, a myriad of analytical and visualisation techniques are employed to generate a visual representation of the data. These often introduce their own uncertainties, in addition to the ones inherent in the data, and these propagated and compounded uncertainties can result in impaired decision making. The user's confidence or trust in the results depends on the extent of user's awareness of the underlying uncertainties generated on the system side. This paper unpacks the uncertainties that propagate through visual analytics systems, illustrates how human's perceptual and cognitive biases influence the user's awareness of such uncertainties, and how this affects the user's trust building. The knowledge generation model for visual analytics is used to provide a terminology and framework to discuss the consequences of these aspects in knowledge construction and though examples, machine uncertainty is compared to human trust measures with provenance. Furthermore, guidelines for the design of uncertainty-aware systems are presented that can aid the user in better decision making.

Visual Encodings of Temporal Uncertainty: A Comparative User Study

A number of studies have investigated different ways of visualizing uncertainty. However, in the temporal dimension, it is still an open question how to best represent uncertainty, since the special characteristics of time require special visual encodings and may provoke different interpretations. Thus, we have conducted a comprehensive study comparing alternative visual encodings of intervals with uncertain start and end times: gradient plots, violin plots, accumulated probability plots, error bars, centered error bars, and ambiguation. Our results reveal significant differences in error rates and completion time for these different visualization types and different tasks. We recommend using ambiguation - using a lighter color value to represent uncertain regions - or error bars for judging durations and temporal bounds, and gradient plots - using fading color or transparency - for judging probability values.

Visually Comparing Weather Features in Forecasts

Meteorologists process and analyze weather forecasts using visualization in order to examine the behaviors of and relationships among weather features. In this design study conducted with meteorologists in decision support roles, we identified and attempted to address two significant common challenges in weather visualization: the employment of inconsistent and often ineffective visual encoding practices across a wide range of visualizations, and a lack of support for directly visualizing how different weather features relate across an ensemble of possible forecast outcomes. In this work, we present a characterization of the problems and data associated with meteorological forecasting, we propose a set of informed default encoding choices that integrate existing meteorological conventions with effective visualization practice, and we extend a set of techniques as an initial step toward directly visualizing the interactions of multiple features over an ensemble forecast. We discuss the integration of these contributions into a functional prototype tool, and also reflect on the many practical challenges that arise when working with weather data.

Representing Uncertainty in Graph Edges: An Evaluation of Paired Visual Variables

When visualizing data with uncertainty, a common approach is to treat uncertainty as an additional dimension and encode it using a visual variable. The effectiveness of this approach depends on how the visual variables chosen for representing uncertainty and other attributes interact to influence the user's perception of each variable. We report a user study on the perception of graph edge attributes when uncertainty associated with each edge and the main edge attribute are visualized simultaneously using two separate visual variables. The study covers four visual variables that are commonly used for visualizing uncertainty on line graphical primitives: lightness, grain, fuzziness, and transparency. We select width, hue, and saturation for visualizing the main edge attribute and hypothesize that we can observe interference between the visual variable chosen to encode the main edge attribute and that to encode uncertainty, as suggested by the concept of dimensional integrality. Grouping the seven visual variables as color-based, focus-based, or geometry-based, we further hypothesize that the degree of interference is affected by the groups to which the two visual variables belong. We consider two further factors in the study: discriminability level for each visual variable as a factor intrinsic to the visual variables and graph-task type (visual search versus comparison) as a factor extrinsic to the visual variables. Our results show that the effectiveness of a visual variable in depicting uncertainty is strongly mediated by all the factors examined here. Focus-based visual variables (fuzziness, grain, and transparency) are robust to the choice of visual variables for encoding the main edge attribute, though fuzziness has stronger negative impact on the perception of width and transparency has stronger negative impact on the perception of hue than the other uncertainty visual variables. We found that interference between hue and lightness is much greater than that between saturation and lightness, though all three are color-based visual variables. We also found a compound relationship between discriminability level and the degree of dimensional integrality. We discuss the generalizability and limitation of the results and conclude with design considerations for visualizing graph uncertainty derived from these results, including recommended choices of visual variables when the relative importance of data attributes and graph tasks is known.

Urban heat indicator map for climate adaptation planning

By 2050, 75 % of the world's population will live in cities and the occurrence of heat wave events might have doubled. Mapping the climate and land use change impact for urban heat events should set the agenda for adaptation planning at the local scale. Literature on urban heat mapping does not reveal a clear indicator to visualise the urban heat impacts that includes consequences of land use and climate changes for planning purposes. This paper introduces a stepwise approach to develop a single complex indicator to map the urban heat impact for local climate adaptation planning processes. Information on climatic drivers and land use characteristics are combined and projected for future land use and climate change impacts. Next, several visualisation techniques are developed to investigate which techniques are most effective to visualise complex information with multiple variables in one visualisation. A usability test is performed to investigate how indicator and map meet the information and communication needs of policy makers. Our findings reveal that it is important to add information on future impacts to set the agenda for adaptation planning at the local scale. Applying cartographic techniques in a map series presentation has proven to be effective to map complex information in a single image and fulfil most of the identified information needs. Based on our finding, we introduce the information enrichment chain as a promising approach to support local adaptation planning.

Testing Map Features Designed to Convey the Uncertainty of Cancer Risk: Insights Gained From Assessing Judgments of Information Adequacy and Communication Goals

Barriers to communicating the uncertainty of environmental health risks include preferences for certain information and low numeracy. Map features designed to communicate the magnitude and uncertainty of estimated cancer risk from air pollution were tested among 826 participants to assess how map features influenced judgments of adequacy and the intended communication goals. An uncertain versus certain visual feature was judged as less adequate but met both communication goals and addressed numeracy barriers. Expressing relative risk using words communicated uncertainty and addressed numeracy barriers but was judged as highly inadequate. Risk communication and visual cognition concepts were applied to explain findings.

Error Bars Considered Harmful: Exploring Alternate Encodings for Mean and Error

When making an inference or comparison with uncertain, noisy, or incomplete data, measurement error and confidence intervals can be as important for judgment as the actual mean values of different groups. These often misunderstood statistical quantities are frequently represented by bar charts with error bars. This paper investigates drawbacks with this standard encoding, and considers a set of alternatives designed to more effectively communicate the implications of mean and error data to a general audience, drawing from lessons learned from the use of visual statistics in the information visualization community. We present a series of crowd-sourced experiments that confirm that the encoding of mean and error significantly changes how viewers make decisions about uncertain data. Careful consideration of design tradeoffs in the visual presentation of data results in human reasoning that is more consistently aligned with statistical inferences. We suggest the use of gradient plots (which use transparency to encode uncertainty) and violin plots (which use width) as better alternatives for inferential tasks than bar charts with error bars.

The Prospects and Problems of Integrating Sketch Maps with Geographic Information Systems to Understand Environmental Perception: A Case Study of Mapping Youth Fear in Los Angeles Gang Neighborhoods

How people feel about places matters, especially in their neighborhood. It matters for their health, the health of their children, and their social cohesion and use of local resources. A growing body of research in public health, planning, psychology, and sociology bears out this point. Recently, a new methodological tack has been taken to find out how people feel about places. The sketch map, a once popular tool of behavioral geographers and environmental psychologists to understand how people perceive the structural aspects of places, is now being used in concert with geographic information systems (GIS) to capture and spatially analyze the emotional side of urban environmental perception. This confluence is generating exciting prospects for what we can learn about the characteristics of the urban environment that elicit emotion. However, due to the uncritical way this approach has been employed to date, excitement about the prospects must be tempered by the acknowledgement of its potential problems. In this paper we review the extant research on integrating sketch maps with GIS and then employ a case study of mapping youth fear in Los Angeles gang neighborhoods to demonstrate these prospects and the problems, particularly in the areas of (1) representation of environmental perception in GIS and (2) spatial analysis of these data.

Characterizing and visualizing predictive uncertainty in numerical ensembles through Bayesian model averaging

Numerical ensemble forecasting is a powerful tool that drives many risk analysis efforts and decision making tasks. These ensembles are composed of individual simulations that each uniquely model a possible outcome for a common event of interest: e.g., the direction and force of a hurricane, or the path of travel and mortality rate of a pandemic. This paper presents a new visual strategy to help quantify and characterize a numerical ensemble's predictive uncertainty: i.e., the ability for ensemble constituents to accurately and consistently predict an event of interest based on ground truth observations. Our strategy employs a Bayesian framework to first construct a statistical aggregate from the ensemble. We extend the information obtained from the aggregate with a visualization strategy that characterizes predictive uncertainty at two levels: at a global level, which assesses the ensemble as a whole, as well as a local level, which examines each of the ensemble's constituents. Through this approach, modelers are able to better assess the predictive strengths and weaknesses of the ensemble as a whole, as well as individual models. We apply our method to two datasets to demonstrate its broad applicability.

Bristle Maps: a multivariate abstraction technique for geovisualization

We present Bristle Maps, a novel method for the aggregation, abstraction, and stylization of spatiotemporal data that enables multiattribute visualization, exploration, and analysis. This visualization technique supports the display of multidimensional data by providing users with a multiparameter encoding scheme within a single visual encoding paradigm. Given a set of geographically located spatiotemporal events, we approximate the data as a continuous function using kernel density estimation. The density estimation encodes the probability that an event will occur within the space over a given temporal aggregation. These probability values, for one or more set of events, are then encoded into a bristle map. A bristle map consists of a series of straight lines that extend from, and are connected to, linear map elements such as roads, train, subway lines, and so on. These lines vary in length, density, color, orientation, and transparency—creating the multivariate attribute encoding scheme where event magnitude, change, and uncertainty can be mapped as various bristle parameters. This approach increases the amount of information displayed in a single plot and allows for unique designs for various information schemes. We show the application of our bristle map encoding scheme using categorical spatiotemporal police reports. Our examples demonstrate the use of our technique for visualizing data magnitude, variable comparisons, and a variety of multivariate attribute combinations. To evaluate the effectiveness of our bristle map, we have conducted quantitative and qualitative evaluations in which we compare our bristle map to conventional geovisualization techniques. Our results show that bristle maps are competitive in completion time and accuracy of tasks with various levels of complexity.

Decisions With Uncertainty: The Glass Half Full

Each of us makes important decisions involving uncertainty in domains in which we are not experts, such as retirement planning, medical treatment, and precautions against severe weather. Often, reliable information about uncertainty is available to us, although how effectively we incorporate it into the decision process remains in question. Previous research suggests that people are error-prone when reasoning with probability. However, recent research in weather-related decision making is more encouraging. Unlike earlier work that compares people’s decisions with a rational standard, this research compares decisions made by people with and without uncertainty information. The results suggest that including specific numeric uncertainty estimates in weather forecasts increases trust and gives people a better idea of what to expect in terms of both the range of possible outcomes and the amount of uncertainty in the particular situation, all of which benefit precautionary decisions. However, the advantage for uncertainty estimates depends critically on how they are expressed. It is crucial that the expression is compatible with both the decision task and cognitive processes of the user.

The influence of uncertain map features on risk beliefs and perceived ambiguity for maps of modeled cancer risk from air pollution

Maps are often used to convey information generated by models, for example, modeled cancer risk from air pollution. The concrete nature of images, such as maps, may convey more certainty than warranted for modeled information. Three map features were selected to communicate the uncertainty of modeled cancer risk: (i) map contours appeared in or out of focus, (ii) one or three colors were used, and (iii) a verbal-relative or numeric risk expression was used in the legend. Study aims were to assess how these features influenced risk beliefs and the ambiguity of risk beliefs at four assigned map locations that varied by risk level. We applied an integrated conceptual framework to conduct this full factorial experiment with 32 maps that varied by the three dichotomous features and four risk levels; 826 university students participated. Data was analyzed using structural equation modeling. Unfocused contours and the verbal-relative risk expression generated more ambiguity than their counterparts. Focused contours generated stronger risk beliefs for higher risk levels and weaker beliefs for lower risk levels. Number of colors had minimal influence. The magnitude of risk level, conveyed using incrementally darker shading, had a substantial dose-response influence on the strength of risk beliefs. Personal characteristics of prior beliefs and numeracy also had substantial influences. Bottom-up and top-down information processing suggest why iconic visual features of incremental shading and contour focus had the strongest visual influences on risk beliefs and ambiguity. Variations in contour focus and risk expression show promise for fostering appropriate levels of ambiguity.

Visual Semiotics & Uncertainty Visualization: An Empirical Study

This paper presents two linked empirical studies focused on uncertainty visualization. The experiments are framed from two conceptual perspectives. First, a typology of uncertainty is used to delineate kinds of uncertainty matched with space, time, and attribute components of data. Second, concepts from visual semiotics are applied to characterize the kind of visual signification that is appropriate for representing those different categories of uncertainty. This framework guided the two experiments reported here. The first addresses representation intuitiveness, considering both visual variables and iconicity of representation. The second addresses relative performance of the most intuitive abstract and iconic representations of uncertainty on a map reading task. Combined results suggest initial guidelines for representing uncertainty and discussion focuses on practical applicability of results.

Evaluating Sketchiness as a Visual Variable for the Depiction of Qualitative Uncertainty

We report on results of a series of user studies on the perception of four visual variables that are commonly used in the literature to depict uncertainty. To the best of our knowledge, we provide the first formal evaluation of the use of these variables to facilitate an easier reading of uncertainty in visualizations that rely on line graphical primitives. In addition to blur, dashing and grayscale, we investigate the use of `sketchiness' as a visual variable because it conveys visual impreciseness that may be associated with data quality. Inspired by work in non-photorealistic rendering and by the features of hand-drawn lines, we generate line trajectories that resemble hand-drawn strokes of various levels of proficiency-ranging from child to adult strokes-where the amount of perturbations in the line corresponds to the level of uncertainty in the data. Our results show that sketchiness is a viable alternative for the visualization of uncertainty in lines and is as intuitive as blur; although people subjectively prefer dashing style over blur, grayscale and sketchiness. We discuss advantages and limitations of each technique and conclude with design considerations on how to deploy these visual variables to effectively depict various levels of uncertainty for line marks.

A method for displaying imprecision in early radiological emergency assessments

This paper presents an approach to estimating and displaying the imprecision associated with predictions from early emergency response calculations based on a limited number of off-site measurements and incomplete information about the nature of the release. The method enables key elements of imprecision to be included in the assessment in a simple and transparent manner, notably those arising from alternative weather evolutions, release durations, and factors influencing the amount of radioactivity deposited to ground. The presentation of assessment results incorporating lack of knowledge in a way that is easily understandable is important in the context of emergency decision making, and options for alternative styles of display are presented and discussed. A new system of early emergency response for radiological releases to atmosphere based on this approach is under development.

Noodles: a tool for visualization of numerical weather model ensemble uncertainty

Numerical weather prediction ensembles are routinely used for operational weather forecasting. The members of these ensembles are individual simulations with either slightly perturbed initial conditions or different model parameterizations, or occasionally both. Multi-member ensemble output is usually large, multivariate, and challenging to interpret interactively. Forecast meteorologists are interested in understanding the uncertainties associated with numerical weather prediction; specifically variability between the ensemble members. Currently, visualization of ensemble members is mostly accomplished through spaghetti plots of a single mid-troposphere pressure surface height contour. In order to explore new uncertainty visualization methods, the Weather Research and Forecasting (WRF) model was used to create a 48-hour, 18 member parameterization ensemble of the 13 March 1993 "Superstorm". A tool was designed to interactively explore the ensemble uncertainty of three important weather variables: water-vapor mixing ratio, perturbation potential temperature, and perturbation pressure. Uncertainty was quantified using individual ensemble member standard deviation, inter-quartile range, and the width of the 95% confidence interval. Bootstrapping was employed to overcome the dependence on normality in the uncertainty metrics. A coordinated view of ribbon and glyph-based uncertainty visualization, spaghetti plots, iso-pressure colormaps, and data transect plots was provided to two meteorologists for expert evaluation. They found it useful in assessing uncertainty in the data, especially in finding outliers in the ensemble run and therefore avoiding the WRF parameterizations that lead to these outliers. Additionally, the meteorologists could identify spatial regions where the uncertainty was significantly high, allowing for identification of poorly simulated storm environments and physical interpretation of these model issues.

Value-by-alpha maps: An alternative technique to the cartogram

The cartogram, or value-by-area map, is a popular technique for cartographically representing social data. Such maps visually equalize a basemap prior to mapping a social variable by adjusting the size of each enumeration unit by a second, related variable. However, to scale the basemap units according to an equalizing variable, cartograms must distort the shape and/or topology of the original geography. Such compromises reduce the effectiveness of the visualization for elemental and general map-reading tasks. Here we describe a new kind of representation, termed a value-by-alpha map, which visually equalizes the basemap by adjusting the alpha channel, rather than the size, of each enumeration unit. Although not without its own limitations, the value-by-alpha map is able to circumvent the compromise inherent to the cartogram form, perfectly equalizing the basemap while preserving both shape and topology.

The dynamic process of cognitive mapping in the absence of visual cues: human data compared with animal studies

The present study aimed to investigate the behavior involved in constructing spatial representation in humans. For this, blindfolded adult human subjects were introduced into an unfamiliar environment, where they were requested to move incessantly for 10 min. Analysis of the locomotor activity of the participants revealed the following exploratory behaviors: (1) ;looping'; (2) ;wall-following'; (3) ;step-counting'; (4) ;cross-cutting'; and (5) ;free traveling'. Looping is a typical exploratory mode of sightless explorers, based on returning to a recently traveled place. Wall-following is common in enclosed spaces, whereby explorers follow the perimeter of the environment. Both looping and wall-following are based on an egocentric frame of reference by which explorers obtain information about the shape, size and landmarks in the environment. Blindfolded explorers displayed step-counting in order to scale the environment and the relationships in it. Altogether, exploration by looping, wall-following and step-counting resulted in an allocentric spatial representation. The acquisition of spatial representation was manifested by cross-cutting and free travel, with subjects walking in a relatively fast and decisive manner. In light of the above modes of activity, we suggest that exploration of an unfamiliar environment is a synergetic self-organized process (synergetic inter-representation networks, SIRN model); an interplay between external and internal representations. According to this model, the interplay gives rise to an order parameter, such as the environment's dimensions or geometry, enabling progression to a subsequent exploratory behavior. This dynamic and sequential interplay reaches a steady state when a spatial representation (i.e. ;cognitive map') is established.

A user study to compare four uncertainty visualization methods for 1D and 2D datasets

Many techniques have been proposed to show uncertainty in data visualizations. However, very little is known about their effectiveness in conveying meaningful information. In this paper, we present a user study that evaluates the perception of uncertainty amongst four of the most commonly used techniques for visualizing uncertainty in one-dimensional and two-dimensional data. The techniques evaluated are traditional errorbars, scaled size of glyphs, color-mapping on glyphs, and color-mapping of uncertainty on the data surface. The study uses generated data that was designed to represent the systematic and random uncertainty components. Twenty-seven users performed two types of search tasks and two types of counting tasks on 1D and 2D datasets. The search tasks involved finding data points that were least or most uncertain. The counting tasks involved counting data features or uncertainty features. A 4x4 full-factorial ANOVA indicated a significant interaction between the techniques used and the type of tasks assigned for both datasets indicating that differences in performance between the four techniques depended on the type of task performed. Several one-way ANOVAs were computed to explore the simple main effects. Bonferronni's correction was used to control for the family-wise error rate for alpha-inflation. Although we did not find a consistent order among the four techniques for all the tasks, there are several findings from the study that we think are useful for uncertainty visualization design. We found a significant difference in user performance between searching for locations of high and searching for locations of low uncertainty. Errorbars consistently underperformed throughout the experiment. Scaling the size of glyphs and color-mapping of the surface performed reasonably well. The efficiency of most of these techniques were highly dependent on the tasks performed. We believe that these findings can be used in future uncertainty visualization design. In addition, the framework developed in this user study presents a structured approach to evaluate uncertainty visualization techniques, as well as provides a basis for future research in uncertainty visualization.