Recent evolution of learnability in American English from 1800 to 2000

How cognitive selection affects language change

Like biological species, words in language must compete to survive. Previously, it has been shown that language changes in response to cognitive constraints and over time becomes more learnable. Here, we use two complementary research paradigms to demonstrate how the survival of existing word forms can be predicted by psycholinguistic properties that impact language production. In the first study, we analyzed the survival of words in the context of interpersonal communication. We analyzed data from a large-scale serial-reproduction experiment in which stories were passed down along a transmission chain over multiple participants. The results show that words that are acquired earlier in life, more concrete, more arousing, and more emotional are more likely to survive retellings. We reason that the same trend might scale up to language evolution over multiple generations of natural language users. If that is the case, the same set of psycholinguistic properties should also account for the change of word frequency in natural language corpora over historical time. That is what we found in two large historical-language corpora (Study 2): Early acquisition, concreteness, and high arousal all predict increasing word frequency over the past 200 y. However, the two studies diverge with respect to the impact of word valence and word length, which we take up in the discussion. By bridging micro-level behavioral preferences and macro-level language patterns, our investigation sheds light on the cognitive mechanisms underlying word competition.

On the Connection Between Language Change and Language Processing

Previous studies provided evidence for a connection between language processing and language change. We add to these studies with an exploration of the influence of lexical-distributional properties of words in orthographic space, semantic space, and the mapping between orthographic and semantic space on the probability of lexical extinction. Through a binomial linear regression analysis, we investigated the probability of lexical extinction by the first decade of the twenty-first century (2000s) for words that existed in the first decade of the nineteenth-century (1800s) in eight data sets for five languages: English, French, German, Italian, and Spanish. The binomial linear regression analysis revealed that words that are more similar in form to other words are less likely to disappear from a language. By contrast, words that are more similar in meaning to other words are more likely to become extinct. In addition, a more consistent mapping between form and meaning protects a word from lexical extinction. A nonlinear time-to-event analysis furthermore revealed that the position of a word in orthographic and semantic space continues to influence the probability of it disappearing from a language for at least 200 years. Effects of the lexical-distributional properties of words under investigation here have been reported in the language processing literature as well. The results reported here, therefore, fit well with a usage-based approach to language change, which holds that language change is at least to some extent connected to cognitive mechanisms in the human brain.

Concreteness ratings for 62,000 English multiword expressions

Concreteness describes the degree to which a word's meaning is understood through perception and action. Many studies use the Brysbaert et al. (2014) concreteness ratings to investigate language processing and text analysis. However, these ratings are limited to English single words and a few two-word expressions. Increasingly, attention is focused on the importance of multiword expressions, given their centrality in everyday language use and language acquisition. We present concreteness ratings for 62,889 multiword expressions and examine their relationship to the existing concreteness ratings for single words and two-word expressions. These new ratings represent the first big dataset of multiword expressions, and will be useful for researchers interested in language acquisition and language processing, as well as natural language processing and text analysis.

Diachronic semantic change in language is constrained by how people use and learn language

While it has long been understood that the human mind evolved to learn language, recent studies have begun to ask the inverted question: How has language evolved under the cognitive constraints of its users and become more learnable over time? In this paper, we explored how the semantic change of English words is shaped by the way humans acquire and process language. In Study 1, we quantified the extent of semantic change over the past 200 years and found that meaning change is more likely for words that are acquired later in life and are more difficult to process. We argue that it is human cognition that constrains the semantic evolution of words, rather than the other way around, because historical meanings of words were not easily accessible to people living today, and therefore could not have directly influenced how they learn and process language. In Study 2, we went further to show that semantic change, while bringing the benefit of meeting communicative needs, is cognitively costly for those who were born early enough to experience the change: Semantic change between 1970 and 2000 hindered processing speeds among middle-aged adults (ages 45–55) but not in younger adults (ages <25) in a semantic decision task. This hampering effect may have, in turn, curbed the rate of semantic change so that language does not change too fast for the human mind to catch up. Taken together, our research demonstrates that semantic change is shaped by processing and acquisition patterns across generations of language users.

Moral foundations tracked over 200 years of lexicographic data, and their predictors

The prediction that reduction of negative selection decreases group-level competitiveness, as reflected in increased individual-focused and diminished group-focused moral foundations, is tested. To measure this hypothesized shift in moral foundations, we conduct a culturomic analysis of the utilization frequencies of items sourced from the moral foundations item pool, tracked among Britannic populations from 1800 to 1999 using Google Ngram Viewer. The resultant higher-order factor, which tracks increasing individualizing values and decreasing binding values, is termed Asabiyyah (capturing social cohesion and collective purpose). Two predictors of this factor are examined: change in the strength of intergroup competition and change in levels of indicators of developmental instability. Both the strength of intergroup competition and levels of developmental instability associate with Asabiyyah. Rising developmental instability mediates the impact of inter-group competition, indicating that reduced between-group competition might have relaxed negative selection against mutations, which might reduce Asabiyyah via their effects on inter-genomic transactions. These results must be interpreted carefully, given the clear real-world evidence that explicit commitment to group-oriented values often features in harmful and maladaptive social and political ideologies of an extreme character.

Assessing Lexical Psychological Properties in Second Language Production: A Dynamic Semantic Similarity Approach

Previous studies of the lexical psycholinguistic properties (LPPs) in second language (L2) production have assessed the degree of an LPP dimension of an L2 corpus by computing the mean ratings of unique content words in the corpus for that dimension, without considering the possibility that learners at different proficiency levels may perceive the degree of that dimension of the same words differently. This study extended a dynamic semantic similarity algorithm to estimate the degree of five different LPP dimensions of several sub-corpora of the Education First-Cambridge Open Language Database representing L2 English learners at different proficiency levels. Our findings provide initial evidence for the validity of the algorithm for assessing the LPPs in L2 production and contribute useful insights into between-proficiency relationships and cross-proficiency differences in the LPPs in L2 production as well as the relationships among different LPP dimensions.

Language patterns of outgroup prejudice

Although explicit verbal expression of prejudice and stereotypes may have become less common due to the recent rise of social norms against prejudice, prejudice in language still persists in more subtle forms. It remains unclear whether and how language patterns predict variance in prejudice across a large number of minority groups. Informed by construal level theory, intergroup-contact theory, and linguistic expectancy bias, we leverage a natural language corpus of 1.8 million newspaper articles to investigate patterns of language referencing 60 U.S. minority groups. We found that perception of social distance among immigrant groups is reflected in language production: Groups perceived as socially distant (vs. close) are also more likely to be mentioned in abstract (vs. concrete) language. Concreteness was also strongly positively correlated with sentiment, a phenomenon that was unique to language concerning minority groups, suggesting a strong tendency for more socially distant groups to be represented with more negative language. We also provide a qualitative exploration of the content of outgroup prejudice by applying Latent Dirichlet Allocation to language referencing minority groups in the context of immigration. We identified 15 immigrant-related topics (e.g., politics, arts, crime, illegal workers, museums, food) and the strength of their association and relationship with perceived sentiment for each minority group. This research demonstrates how perceived social distance and language concreteness are related and correlate with outgroup negativity, provides a practical and ecologically valid method for investigating perceptions of minority groups in language, and helps elaborate the connection between theoretical positions from social psychology with recent studies from computer science on prejudice embedded in natural language.

Words with Consistent Diachronic Usage Patterns are Learned Earlier: A Computational Analysis Using Temporally Aligned Word Embeddings

In this study, we use temporally aligned word embeddings and a large diachronic corpus of English to quantify language change in a data-driven, scalable way, which is grounded in language use. We show a unique and reliable relation between measures of language change and age of acquisition (AoA) while controlling for frequency, contextual diversity, concreteness, length, dominant part of speech, orthographic neighborhood density, and diachronic frequency variation. We analyze measures of language change tackling both the change in lexical representations and the change in the relation between lexical representations and the words with the most similar usage patterns, showing that they capture different aspects of language change. Our results show a unique relation between language change and AoA, which is stronger when considering neighborhood-level measures of language change: Words with more coherent diachronic usage patterns tend to be acquired earlier. The results support theories positing a link between ontogenetic and ethnogenetic processes in language.

The evolutionary pattern of language in scientific writings: A case study of Philosophical Transactions of Royal Society (1665–1869)

Scientific writings, as one essential part of human culture, have evolved over centuries into their current form. Knowing how scientific writings evolved is particularly helpful in understanding how trends in scientific culture developed. It also allows us to better understand how scientific culture was interwoven with human culture generally. The availability of massive digitized texts and the progress in computational technologies today provide us with a convenient and credible way to discern the evolutionary patterns in scientific writings by examining the diachronic linguistic changes. The linguistic changes in scientific writings reflect the genre shifts that took place with historical changes in science and scientific writings. This study investigates a general evolutionary linguistic pattern in scientific writings. It does so by merging two credible computational methods: relative entropy; word-embedding concreteness and imageability. It thus creates a novel quantitative methodology and applies this to the examination of diachronic changes in the Philosophical Transactions of Royal Society (PTRS, 1665–1869). The data from two computational approaches can be well mapped to support the argument that this journal followed the evolutionary trend of increasing professionalization and specialization. But it also shows that language use in this journal was greatly influenced by historical events and other socio-cultural factors. This study, as a “culturomic” approach, demonstrates that the linguistic evolutionary patterns in scientific discourse have been interrupted by external factors even though this scientific discourse would likely have cumulatively developed into a professional and specialized genre. The approaches proposed by this study can make a great contribution to full-text analysis in scientometrics.

A brief history of risk

Despite increasing life expectancy and high levels of welfare, health care, and public safety in most post-industrial countries, the public discourse often revolves around perceived threats. Terrorism, global pandemics, and environmental catastrophes are just a few of the risks that dominate media coverage. Is this public discourse on risk disconnected from reality? To examine this issue, we analyzed the dynamics of the risk discourse in two natural language text corpora. Specifically, we tracked latent semantic patterns over a period of 150 years to address four questions: First, we examined how the frequency of the word risk has changed over historical time. Is the construct of risk playing an ever-increasing role in the public discourse, as the sociological notion of a 'risk society' suggests? Second, we investigated how the sentiments for the words co-occurring with risk have changed. Are the connotations of risk becoming increasingly ominous? Third, how has the meaning of risk changed relative to close associates such as danger and hazard? Is risk more subject to semantic change? Finally, we decompose the construct of risk into the specific topics with which it has been associated and track those topics over historical time. This brief history of the semantics of risk reveals new and surprising insights-a fourfold increase in frequency, increasingly negative sentiment, a semantic drift toward forecasting and prevention, and a shift away from war toward chronic disease-reflecting the conceptual evolution of risk in the archeological records of public discourse.

The Macroscope: A tool for examining the historical structure of language

The recent rise in digitized historical text has made it possible to quantitatively study our psychological past. This involves understanding changes in what words meant, how words were used, and how these changes may have responded to changes in the environment, such as in healthcare, wealth disparity, and war. Here we make available a tool, the Macroscope, for studying historical changes in language over the last two centuries. The Macroscope uses over 155 billion words of historical text, which will grow as we include new historical corpora, and derives word properties from frequency-of-usage and co-occurrence patterns over time. Using co-occurrence patterns, the Macroscope can track changes in semantics, allowing researchers to identify semantically stable and unstable words in historical text and providing quantitative information about changes in a word’s valence, arousal, and concreteness, as well as information about new properties, such as semantic drift. The Macroscope provides information about both the local and global properties of words, as well as information about how these properties change over time, allowing researchers to visualize and download data in order to make inferences about historical psychology. Although quantitative historical psychology represents a largely new field of study, we see this work as complementing a wealth of other historical investigations, offering new insights and new approaches to understanding existing theory. The Macroscope is available online at http://www.macroscope.tech.

Historical analysis of national subjective wellbeing using millions of digitized books

In addition to improving quality of life, higher subjective wellbeing leads to fewer health problems and higher productivity, making subjective wellbeing a focal issue among researchers and governments. However, it is difficult to estimate how happy people were during previous centuries. Here we show that a method based on the quantitative analysis of natural language published over the past 200 years captures reliable patterns in historical subjective wellbeing. Using sentiment analysis on the basis of psychological valence norms, we compute a national valence index for the United Kingdom, the United States, Germany and Italy, indicating relative happiness in response to national and international wars and in comparison to historical trends in longevity and gross domestic product. We validate our method using Eurobarometer survey data from the 1970s and demonstrate robustness using words with stable historical meanings, diverse corpora (newspapers, magazines and books) and additional word norms. By providing a window on quantitative historical psychology, this approach could inform policy and economic history.

A model of contact-induced language change: Testing the role of second language speakers in the evolution of Mozambican Portuguese

Language change is accelerated by language contact, especially by contact that occurs when a group of speakers shifts from one language to another. This has commonly been explained by linguistic innovation occurring during second language acquisition. This hypothesis is based on historical reconstructions of instances of contact and has not been formally tested on empirical data. In this paper, we construct an agent-based model to formalize the hypothesis that second language speakers are responsible for accelerated language change during language shift. We compare model predictions to a unique combination of diachronic linguistic and demographic data from Maputu, Mozambique. The model correctly predicts an increased proportional use of the novel linguistic variants during the period we study. We find that a modified version of the model is a better fit to one of our two datasets and discuss plausible reasons for this. As a general conclusion concerning typological differences between contact-induced and non-contact-induced language change, we suggest that multiple introductions of a new linguistic variant by different individuals may be the mechanism by which language contact accelerates language change.

Guideline for improving the reliability of Google Ngram studies: Evidence from religious terms

The Google Books Ngram Viewer (Google Ngram) is a search engine that charts word frequencies from a large corpus of books and thereby allows for the examination of cultural change as it is reflected in books. While the tool's massive corpus of data (about 8 million books or 6% of all books ever published) has been used in various scientific studies, concerns about the accuracy of results have simultaneously emerged. This paper reviews the literature and serves as a guideline for improving Google Ngram studies by suggesting five methodological procedures suited to increase the reliability of results. In particular, we recommend the use of (I) different language corpora, (II) cross-checks on different corpora from the same language, (III) word inflections, (IV) synonyms, and (V) a standardization procedure that accounts for both the influx of data and unequal weights of word frequencies. Further, we outline how to combine these procedures and address the risk of potential biases arising from censorship and propaganda. As an example of the proposed procedures, we examine the cross-cultural expression of religion via religious terms for the years 1900 to 2000. Special emphasis is placed on the situation during World War II. In line with the strand of literature that emphasizes the decline of collectivistic values, our results suggest an overall decrease of religion's importance. However, religion re-gains importance during times of crisis such as World War II. By comparing the results obtained through the different methods, we illustrate that applying and particularly combining our suggested procedures increase the reliability of results and prevents authors from deriving wrong assumptions.

Humor norms for 4,997 English words

Humor ratings are provided for 4,997 English words collected from 821 participants using an online crowd-sourcing platform. Each participant rated 211 words on a scale from 1 (humorless) to 5 (humorous). To provide for comparisons across norms, words were chosen from a set common to a number of previously collected norms (e.g., arousal, valence, dominance, concreteness, age of acquisition, and reaction time). The complete dataset provides researchers with a list of humor ratings and includes information on gender, age, and educational differences. Results of analyses show that the ratings have reliability on a par with previous ratings and are not well predicted by existing norms.

Simpler grammar, larger vocabulary: How population size affects language

Languages with many speakers tend to be structurally simple while small communities sometimes develop languages with great structural complexity. Paradoxically, the opposite pattern appears to be observed for non-structural properties of language such as vocabulary size. These apparently opposite patterns pose a challenge for theories of language change and evolution. We use computational simulations to show that this inverse pattern can depend on a single factor: ease of diffusion through the population. A population of interacting agents was arranged on a network, passing linguistic conventions to one another along network links. Agents can invent new conventions, or replicate conventions that they have previously generated themselves or learned from other agents. Linguistic conventions are either Easy or Hard to diffuse, depending on how many times an agent needs to encounter a convention to learn it. In large groups, only linguistic conventions that are easy to learn, such as words, tend to proliferate, whereas small groups where everyone talks to everyone else allow for more complex conventions, like grammatical regularities, to be maintained. Our simulations thus suggest that language, and possibly other aspects of culture, may become simpler at the structural level as our world becomes increasingly interconnected.

The Dark Side of Information Proliferation

There are well-understood psychological limits on our capacity to process information. As information proliferation-the consumption and sharing of information-increases through social media and other communications technology, these limits create an attentional bottleneck, favoring information that is more likely to be searched for, attended to, comprehended, encoded, and later reproduced. In information-rich environments, this bottleneck influences the evolution of information via four forces of cognitive selection, selecting for information that is belief-consistent, negative, social, and predictive. Selection for belief-consistent information leads balanced information to support increasingly polarized views. Selection for negative information amplifies information about downside risks and crowds out potential benefits. Selection for social information drives herding, impairs objective assessments, and reduces exploration for solutions to hard problems. Selection for predictive patterns drives overfitting, the replication crisis, and risk seeking. This article summarizes the negative implications of these forces of cognitive selection and presents eight warnings that represent severe pitfalls for the naive "informavore," accelerating extremism, hysteria, herding, and the proliferation of misinformation.

Word Forms Are Structured for Efficient Use

Zipf famously stated that, if natural language lexicons are structured for efficient communication, the words that are used the most frequently should require the least effort. This observation explains the famous finding that the most frequent words in a language tend to be short. A related prediction is that, even within words of the same length, the most frequent word forms should be the ones that are easiest to produce and understand. Using orthographics as a proxy for phonetics, we test this hypothesis using corpora of 96 languages from Wikipedia. We find that, across a variety of languages and language families and controlling for length, the most frequent forms in a language tend to be more orthographically well-formed and have more orthographic neighbors than less frequent forms. We interpret this result as evidence that lexicons are structured by language usage pressures to facilitate efficient communication.

Bad News Has Wings: Dread Risk Mediates Social Amplification in Risk Communication

Social diffusion of information amplifies risk through processes of birth, death, and distortion of message content. Dread risk-involving uncontrollable, fatal, involuntary, and catastrophic outcomes (e.g., terrorist attacks and nuclear accidents)-may be particularly susceptible to amplification because of the psychological biases inherent in dread risk avoidance. To test this, initially balanced information about high or low dread topics was given to a set of individuals who then communicated this information through diffusion chains, each person passing a message to the next. A subset of these chains were also reexposed to the original information. We measured prior knowledge, perceived risk before and after transmission, and, at each link, number of positive and negative statements. Results showed that the more a message was transmitted the more negative statements it contained. This was highest for the high dread topic. Increased perceived risk and production of negative messages was closely related to the amount of negative information that was received, with domain knowledge mitigating this effect. Reexposure to the initial information was ineffectual in reducing bias, demonstrating the enhanced danger of socially transmitted information.

Sequence Memory Constraints Give Rise to Language-Like Structure through Iterated Learning

Human language is composed of sequences of reusable elements. The origins of the sequential structure of language is a hotly debated topic in evolutionary linguistics. In this paper, we show that sets of sequences with language-like statistical properties can emerge from a process of cultural evolution under pressure from chunk-based memory constraints. We employ a novel experimental task that is non-linguistic and non-communicative in nature, in which participants are trained on and later asked to recall a set of sequences one-by-one. Recalled sequences from one participant become training data for the next participant. In this way, we simulate cultural evolution in the laboratory. Our results show a cumulative increase in structure, and by comparing this structure to data from existing linguistic corpora, we demonstrate a close parallel between the sets of sequences that emerge in our experiment and those seen in natural language.

Exploration and exploitation of Victorian science in Darwin's reading notebooks

Search in an environment with an uncertain distribution of resources involves a trade-off between exploitation of past discoveries and further exploration. This extends to information foraging, where a knowledge-seeker shifts between reading in depth and studying new domains. To study this decision-making process, we examine the reading choices made by one of the most celebrated scientists of the modern era: Charles Darwin. From the full-text of books listed in his chronologically-organized reading journals, we generate topic models to quantify his local (text-to-text) and global (text-to-past) reading decisions using Kullback-Liebler Divergence, a cognitively-validated, information-theoretic measure of relative surprise. Rather than a pattern of surprise-minimization, corresponding to a pure exploitation strategy, Darwin's behavior shifts from early exploitation to later exploration, seeking unusually high levels of cognitive surprise relative to previous eras. These shifts, detected by an unsupervised Bayesian model, correlate with major intellectual epochs of his career as identified both by qualitative scholarship and Darwin's own self-commentary. Our methods allow us to compare his consumption of texts with their publication order. We find Darwin's consumption more exploratory than the culture's production, suggesting that underneath gradual societal changes are the explorations of individual synthesis and discovery. Our quantitative methods advance the study of cognitive search through a framework for testing interactions between individual and collective behavior and between short- and long-term consumption choices. This novel application of topic modeling to characterize individual reading complements widespread studies of collective scientific behavior.

Birth of the cool: a two-centuries decline in emotional expression in Anglophone fiction

The presence of emotional words and content in stories has been shown to enhance a story's memorability, and its cultural success. Yet, recent cultural trends run in the opposite direction. Using the Google Books corpus, coupled with two metadata-rich corpora of Anglophone fiction books, we show a decrease in emotionality in English-speaking literature starting plausibly in the nineteenth century. We show that this decrease cannot be explained by changes unrelated to emotionality (such as demographic dynamics concerning age or gender balance, changes in vocabulary richness, or changes in the prevalence of literary genres), and that, in our three corpora, the decrease is driven almost entirely by a decline in the proportion of positive emotion-related words, while the frequency of negative emotion-related words shows little if any decline. Consistently with previous studies, we also find a link between ageing and negative emotionality at the individual level.

Population Size Predicts Lexical Diversity, but so Does the Mean Sea Level --Why It Is Important to Correctly Account for the Structure of Temporal Data

In order to demonstrate why it is important to correctly account for the (serial dependent) structure of temporal data, we document an apparently spectacular relationship between population size and lexical diversity: for five out of seven investigated languages, there is a strong relationship between population size and lexical diversity of the primary language in this country. We show that this relationship is the result of a misspecified model that does not consider the temporal aspect of the data by presenting a similar but nonsensical relationship between the global annual mean sea level and lexical diversity. Given the fact that in the recent past, several studies were published that present surprising links between different economic, cultural, political and (socio-)demographical variables on the one hand and cultural or linguistic characteristics on the other hand, but seem to suffer from exactly this problem, we explain the cause of the misspecification and show that it has profound consequences. We demonstrate how simple transformation of the time series can often solve problems of this type and argue that the evaluation of the plausibility of a relationship is important in this context. We hope that our paper will help both researchers and reviewers to understand why it is important to use special models for the analysis of data with a natural temporal ordering.

Semantic Factors Predict the Rate of Lexical Replacement of Content Words

The rate of lexical replacement estimates the diachronic stability of word forms on the basis of how frequently a proto-language word is replaced or retained in its daughter languages. Lexical replacement rate has been shown to be highly related to word class and word frequency. In this paper, we argue that content words and function words behave differently with respect to lexical replacement rate, and we show that semantic factors predict the lexical replacement rate of content words. For the 167 content items in the Swadesh list, data was gathered on the features of lexical replacement rate, word class, frequency, age of acquisition, synonyms, arousal, imageability and average mutual information, either from published databases or gathered from corpora and lexica. A linear regression model shows that, in addition to frequency, synonyms, senses and imageability are significantly related to the lexical replacement rate of content words–in particular the number of synonyms that a word has. The model shows no differences in lexical replacement rate between word classes, and outperforms a model with word class and word frequency predictors only.