How Communication Can Make Voters Choose Less Well

Individuals, Collectives, and Individuals in Collectives: The Ineliminable Role of Dependence

Our beliefs are inextricably shaped through communication with others. Furthermore, even conversation we conduct in pairs may itself be taking place across a wider, connected social network. Our communications, and with that our thoughts, are consequently typically those of individuals in collectives. This has fundamental consequences with respect to how our beliefs are shaped. This article examines the role of dependence on our beliefs and seeks to demonstrate its importance with respect to key phenomena involving collectives that have been taken to indicate irrationality. It is argued that (with the benefit of hindsight) these phenomena no longer seem surprising when one considers the multiple dependencies that govern information acquisition and the evaluation of cognitive agents in their normal (i.e., social) context.

Knowledge through social networks: Accuracy, error, and polarisation

This paper examines the fundamental problem of testimony. Much of what we believe to know we know in good part, or even entirely, through the testimony of others. The problem with testimony is that we often have very little on which to base estimates of the accuracy of our sources. Simulations with otherwise optimal agents examine the impact of this for the accuracy of our beliefs about the world. It is demonstrated both where social networks of information dissemination help and where they hinder. Most importantly, it is shown that both social networks and a common strategy for gauging the accuracy of our sources give rise to polarisation even for entirely accuracy motivated agents. Crucially these two factors interact, amplifying one another's negative consequences, and this side effect of communication in a social network increases with network size. This suggests a new causal mechanism by which social media may have fostered the increase in polarisation currently observed in many parts of the world.

Beyond collective intelligence: Collective adaptation

We develop a conceptual framework for studying collective adaptation in complex socio-cognitive systems, driven by dynamic interactions of social integration strategies, social environments and problem structures. Going beyond searching for 'intelligent' collectives, we integrate research from different disciplines and outline modelling approaches that can be used to begin answering questions such as why collectives sometimes fail to reach seemingly obvious solutions, how they change their strategies and network structures in response to different problems and how we can anticipate and perhaps change future harmful societal trajectories. We discuss the importance of considering path dependence, lack of optimization and collective myopia to understand the sometimes counterintuitive outcomes of collective adaptation. We call for a transdisciplinary, quantitative and societally useful social science that can help us to understand our rapidly changing and ever more complex societies, avoid collective disasters and reach the full potential of our ability to organize in adaptive collectives.

A variational-autoencoder approach to solve the hidden profile task in hybrid human-machine teams

Algorithmic agents, popularly known as bots, have been accused of spreading misinformation online and supporting fringe views. Collectives are vulnerable to hidden-profile environments, where task-relevant information is unevenly distributed across individuals. To do well in this task, information aggregation must equally weigh minority and majority views against simple but inefficient majority-based decisions. In an experimental design, human volunteers working in teams of 10 were asked to solve a hidden-profile prediction task. We trained a variational auto-encoder (VAE) to learn people’s hidden information distribution by observing how people’s judgments correlated over time. A bot was designed to sample responses from the VAE latent embedding to selectively support opinions proportionally to their under-representation in the team. We show that the presence of a single bot (representing 10% of team members) can significantly increase the polarization between minority and majority opinions by making minority opinions less prone to social influence. Although the effects on hybrid team performance were small, the bot presence significantly influenced opinion dynamics and individual accuracy. These findings show that self-supervized machine learning techniques can be used to design algorithms that can sway opinion dynamics and group outcomes.

Collectives and Epistemic Rationality

Consideration of collectives raises important questions about human rationality. This has long been known for questions about preferences, but it holds also with respect to beliefs. For one, there are contexts (such as voting) where we might care as much, or more, about the rationality of a collective than the rationality of the individuals it comprises. Here, a given standard may yield competing assessments at the individual and the collective level, thus giving rise to important normative questions. At the same time, seemingly rational strategies of individuals may have surprising consequences, or even fail, when exercised by individuals within collectives. This paper will illustrate these considerations with examples, provide an overview of different formal frameworks for understanding and assessing the beliefs of collectives, and it will illustrate how such frameworks can combine with simulations in order to elucidate epistemic norms.

Sensitivity to Evidential Dependencies in Judgments Under Uncertainty

According to Bayesian models of judgment, testimony from independent informants has more evidential value than dependent testimony. Three experiments investigated learners' sensitivity to this distinction. Each experiment used a social version of the balls‐and‐urns task, in which participants judged which of two urns was the most likely source of evidence presented by multiple informants. Informants either provided independent testimony based solely on their own observations or dependent‐sequential testimony that considered the testimonies of previous informants. Although participants updated their beliefs with additional evidence, this updating was generally insensitive to evidential dependency (Experiments 1 and 2). A notable exception was when individuals were separated according to their beliefs about the relative value of independent and sequential evidence. Those who viewed independent evidence as having greater value subsequently gave more weight to independent testimony in the balls‐and‐urns task (Experiment 3), in line with the predictions of a Bayesian model. Our findings suggest that only a minority of individuals conform to Bayesian predictions in the relative weighting of independent and dependent evidence in judgments under uncertainty.

Getting to the source of the illusion of consensus

Consensus between informants is a valuable cue to a claim's epistemic value, when informants' beliefs are developed independently of each other. Recent work (Yousif et al., 2019) described an illusion of consensus such that people did not generally discriminate between the epistemic warrant of true consensus, where a majority claim is supported by multiple independent sources, and false consensus arising from repetition of a single source's claim. Four experiments tested a novel account of the illusion of consensus; that it arises when people are unsure about the independence of the primary sources on which informant claims are based. When this independence relationship was ambiguous we found evidence for the illusion. However, when steps were taken to highlight the independence between data sources in the true consensus conditions, and confidence in a claim was measured against a no consensus baseline (where there was an equal number of reports supporting and opposing a claim), more weight was given to claims based on true consensus than false consensus. These findings show that although the illusion of consensus is prevalent, people do have the capacity to distinguish between true and false consensus.

Does discussion make crowds any wiser?

Does discussion in large groups help or hinder the wisdom of crowds? To give rise to the wisdom of crowds, by which large groups can yield surprisingly accurate answers, aggregation mechanisms such as averaging of opinions or majority voting rely on diversity of opinions, and independence between the voters. Discussion tends to reduce diversity and independence. On the other hand, discussion in small groups has been shown to improve the accuracy of individual answers. To test the effects of discussion in large groups, we gave groups of participants (N = 1958 participants in groups of size ranging from 22 to 212; mean 59) one of three types of problems (demonstrative, factual, ethical) to solve, first individually, and then through discussion. For demonstrative (logical or mathematical) problems, discussion improved individual answers, as well as the answers reached through aggregation. For factual problems, discussion improved individual answers, and either improved or had no effect on the answers reached through aggregation. Our results suggest that, for problems which have a correct answer, discussion in large groups does not detract from the effects of the wisdom of crowds, and tends on the contrary to improve on it.

The Diversity Gap: When Diversity Matters for Knowledge

Can diversity make for better science? Although diversity has ethical and political value, arguments for its epistemic value require a bridge between normative and mechanistic considerations, demonstrating why and how diversity benefits collective intelligence. However, a major hurdle is that the benefits themselves are rather mixed: Quantitative evidence from psychology and behavioral sciences sometimes shows a positive epistemic effect of diversity, but often shows a null effect, or even a negative effect. Here we argue that to make progress with these why and how questions, we need first to rethink when one ought to expect a benefit of cognitive diversity. In doing so, we highlight that the benefits of cognitive diversity are not equally distributed about collective intelligence tasks and are best seen for complex, multistage, creative problem solving, during problem posing and hypothesis generation. Throughout, we additionally outline a series of mechanisms relating diversity and problem complexity, and show how this perspective can inform metascience questions.

A Brief Taxonomy of Hybrid Intelligence

As artificial intelligence becomes ubiquitous in our lives, so do the opportunities to combine machine and human intelligence to obtain more accurate and more resilient prediction models across a wide range of domains. Hybrid intelligence can be designed in many ways, depending on the role of the human and the algorithm in the hybrid system. This paper offers a brief taxonomy of hybrid intelligence, which describes possible relationships between human and machine intelligence for robust forecasting. In this taxonomy, biological intelligence represents one axis of variation, going from individual intelligence (one individual in isolation) to collective intelligence (several connected individuals). The second axis of variation represents increasingly sophisticated algorithms that can take into account more aspects of the forecasting system, from information to task to human problem-solvers. The novelty of the paper lies in the interpretation of recent studies in hybrid intelligence as precursors of a set of algorithms that are expected to be more prominent in the future. These algorithms promise to increase hybrid system’s resilience across a wide range of human errors and biases thanks to greater human-machine understanding. This work ends with a short overview for future research in this field.

Regularized Chained Deep Neural Network Classifier for Multiple Annotators

The increasing popularity of crowdsourcing platforms, i.e., Amazon Mechanical Turk, changes how datasets for supervised learning are built. In these cases, instead of having datasets labeled by one source (which is supposed to be an expert who provided the absolute gold standard), databases holding multiple annotators are provided. However, most state-of-the-art methods devoted to learning from multiple experts assume that the labeler’s behavior is homogeneous across the input feature space. Besides, independence constraints are imposed on annotators’ outputs. This paper presents a regularized chained deep neural network to deal with classification tasks from multiple annotators. The introduced method, termed RCDNN, jointly predicts the ground truth label and the annotators’ performance from input space samples. In turn, RCDNN codes interdependencies among the experts by analyzing the layers’ weights and includes l1, l2, and Monte-Carlo Dropout-based regularizers to deal with the over-fitting issue in deep learning models. Obtained results (using both simulated and real-world annotators) demonstrate that RCDNN can deal with multi-labelers scenarios for classification tasks, defeating state-of-the-art techniques.

Adding Types, But Not Tokens, Affects Property Induction

The extent to which we generalize a novel property from a sample of familiar instances to novel instances depends on the sample composition. Previous property induction experiments have only used samples consisting of novel types (unique entities). Because real-world evidence samples often contain redundant tokens (repetitions of the same entity), we studied the effects on property induction of adding types and tokens to an observed sample. In Experiments 1-3, we presented participants with a sample of birds or flowers known to have a novel property and probed whether this property generalized to novel items varying in similarity to the initial sample. Increasing the number of novel types (e.g., new birds with the target property) in a sample produced tightening, promoting property generalization to highly similar stimuli but decreasing generalization to less similar stimuli. On the other hand, increasing the number of tokens (e.g., repeated presentations of the same bird with the target property) had little effect on generalization. Experiment 4 showed that repeated tokens are encoded and can benefit recognition, but appear to be given little weight when inferring property generalization. We modified an existing Bayesian model of induction (Navarro, Dry, & Lee, 2012) to account for both the information added by new types and the discounting of information conveyed by tokens.

Dependencies in evidential reports: The case for informational advantages

Whether assessing the accuracy of expert forecasting, the pros and cons of group communication, or the value of evidence in diagnostic or predictive reasoning, dependencies between experts, group members, or evidence have traditionally been seen as a form of redundancy. We demonstrate that this conception of dependence conflates the structure of a dependency network, and the observations across this network. By disentangling these two elements we show, via mathematical proof and specific examples, that there are cases where dependencies yield an informational advantage over independence. More precisely, when a structural dependency exists, but observations are either partial or contradicting, these observations provide more support to a hypothesis than when this structural dependency does not exist, ceteris paribus. Furthermore, we show that lay reasoners endorse sufficient assumptions underpinning these advantageous structures yet fail to appreciate their implications for probability judgments and belief revision.

Widening Access to Bayesian Problem Solving

Bayesian reasoning and decision making is widely considered normative because it minimizes prediction error in a coherent way. However, it is often difficult to apply Bayesian principles to complex real world problems, which typically have many unknowns and interconnected variables. Bayesian network modeling techniques make it possible to model such problems and obtain precise predictions about the causal impact that changing the value of one variable may have on the values of other variables connected to it. But Bayesian modeling is itself complex, and has until now remained largely inaccessible to lay people. In a large scale lab experiment, we provide proof of principle that a Bayesian network modeling tool, adapted to provide basic training and guidance on the modeling process to beginners without requiring knowledge of the mathematical machinery working behind the scenes, significantly helps lay people find normative Bayesian solutions to complex problems, compared to generic training on probabilistic reasoning. We discuss the implications of this finding for the use of Bayesian network software tools in applied contexts such as security, medical, forensic, economic or environmental decision making.