A step-by-step guide to collecting and analyzing long-format speech environment (LFSE) recordings

Establishing the reliability of metrics extracted from long-form recordings using LENA and the ACLEW pipeline

Long-form audio recordings are increasingly used to study individual variation, group differences, and many other topics in theoretical and applied fields of developmental science, particularly for the description of children's language input (typically speech from adults) and children's language output (ranging from babble to sentences). The proprietary LENA software has been available for over a decade, and with it, users have come to rely on derived metrics like adult word count (AWC) and child vocalization counts (CVC), which have also more recently been derived using an open-source alternative, the ACLEW pipeline. Yet, there is relatively little work assessing the reliability of long-form metrics in terms of the stability of individual differences across time. Filling this gap, we analyzed eight spoken-language datasets: four from North American English-learning infants, and one each from British English-, French-, American English-/Spanish-, and Quechua-/Spanish-learning infants. The audio data were analyzed using two types of processing software: LENA and the ACLEW open-source pipeline. When all corpora were included, we found relatively low to moderate reliability (across multiple recordings, intraclass correlation coefficient attributed to the child identity [Child ICC], was < 50% for most metrics). There were few differences between the two pipelines. Exploratory analyses suggested some differences as a function of child age and corpora. These findings suggest that, while reliability is likely sufficient for various group-level analyses, caution is needed when using either LENA or ACLEW tools to study individual variation. We also encourage improvement of extant tools, specifically targeting accurate measurement of individual variation.

Language environment and early language production in Slovenian infants: An exploratory study using daylong recordings

Daylong recordings provide an ecologically valid option for analyzing language input, and have become a central method for studying child language development. However, the vast majority of this work has been conducted in North America. We harnessed a unique collection of daylong recordings from Slovenian infants (age: 16-30 months, N = 40, 18 girls), and focus our attention on manually annotated measures of parentese (infant-directed speech with a higher pitch, slower tempo, and exaggerated intonation), conversational turns, infant words, and word combinations. Measures from daylong recordings showed large variation, but were comparable to previous studies with North American samples. Infants heard almost twice as much speech and parentese from mothers compared to fathers, but there were no differences in language input to boys and girls. Positive associations were found between the social-interactional features of language input (parentese, turn-taking) and infants' concurrent language production. Measures of child speech from daylong recordings were positively correlated with measures obtained through the Slovenian MacArthur-Bates Communicative Development Inventory. These results support the notion that the social-interactional features of parental language input are the foundation of infants' language skills, even in an environment where infants spend much of their waking hours in childcare settings, as they do in Slovenia.

Infant-directed communication: Examining the many dimensions of everyday caregiver-infant interactions

Everyday caregiver-infant interactions are dynamic and multidimensional. However, existing research underestimates the dimensionality of infants' experiences, often focusing on one or two communicative signals (e.g., speech alone, or speech and gesture together). Here, we introduce "infant-directed communication" (IDC): the suite of communicative signals from caregivers to infants including speech, action, gesture, emotion, and touch. We recorded 10 min of at-home play between 44 caregivers and their 18- to 24-month-old infants from predominantly white, middle-class, English-speaking families in the United States. Interactions were coded for five dimensions of IDC as well as infants' gestures and vocalizations. Most caregivers used all five dimensions of IDC throughout the interaction, and these dimensions frequently overlapped. For example, over 60% of the speech that infants heard was accompanied by one or more non-verbal communicative cues. However, we saw marked variation across caregivers in their use of IDC, likely reflecting tailored communication to the behaviors and abilities of their infant. Moreover, caregivers systematically increased the dimensionality of IDC, using more overlapping cues in response to infant gestures and vocalizations, and more IDC with infants who had smaller vocabularies. Understanding how and when caregivers use all five signals-together and separately-in interactions with infants has the potential to redefine how developmental scientists conceive of infants' communicative environments, and enhance our understanding of the relations between caregiver input and early learning. RESEARCH HIGHLIGHTS: Infants' everyday interactions with caregivers are dynamic and multimodal, but existing research has underestimated the multidimensionality (i.e., the diversity of simultaneously occurring communicative cues) inherent in infant-directed communication. Over 60% of the speech that infants encounter during at-home, free play interactions overlap with one or more of a variety of non-speech communicative cues. The multidimensionality of caregivers' communicative cues increases in response to infants' gestures and vocalizations, providing new information about how infants' own behaviors shape their input. These findings emphasize the importance of understanding how caregivers use a diverse set of communicative behaviors-both separately and together-during everyday interactions with infants.

Estimating infants' language exposure: A comparison of random and volume sampling from daylong recordings collected in a bilingual community

In North America, the characteristics of a child's language environment predict language outcomes. For example, differences in bilingual language exposure, exposure to electronic media, and exposure to child-directed speech (CDS) relate to children's language growth. Recently, these predictors have been studied through the use of daylong recordings, followed by manual annotation of audio samples selected from these recordings. Using a dataset of daylong recordings collected from bilingually raised infants in the United States as an example, we ask whether two of the most commonly used sampling methods, random sampling and sampling based on high adult speech, differ from each other with regard to estimating the frequencies of specific language behaviors. Daylong recordings from 37 Spanish-English speaking families with infants between 4 and 22 months of age were analyzed. From each child's recording, samples were extracted in two ways (at random/based on high adult speech) and then annotated for Language (Spanish/English/Mixed), CDS, Electronic Media, Social Context, Turn-Taking, and Infant Babbling. Correlation and agreement analyses were performed, in addition to paired sample t-tests, to assess how the choice of one or the other sampling method may affect the estimates. For most behaviors studied, correlation and agreement between the two sampling methods was high (Pearson r values between 0.79 and 0.99 for 16 of 17 measures; Intraclass Correlation Coefficient values between 0.78 and 0.99 for 13 of 17 measures). However, interesting between-sample differences also emerged: the degree of language mixing, the amount of CDS, and the number of conversational turns were all significantly higher when sampling was performed based on high adult speech compared to random sampling. By contrast, the presence of electronic media and one-on-one social contexts was higher when sampling was performed at random. We discuss advantages of choosing one sampling technique over the other, depending on the research question and variables at hand.

It takes a village: Caregiver diversity and language contingency in the UK and rural Gambia

INTRODUCTION

There is substantial diversity within and between contexts globally in caregiving practices and family composition, which may have implications for the early interaction's infants engage in. We draw on data from the Brain Imaging for Global Health (BRIGHT, www.globalfnirs.org/the-bright-project) project, which longitudinally examined infants in the UK and in rural Gambia, West Africa. In The Gambia, households are commonly characterized by multigenerational, frequently polygamous family structures, which, in part, is reflected in the diversity of caregivers a child spends time with. In this paper, we aim to 1) evaluate and validate the Language Environment Analysis (LENA) for use in the Mandinka speaking families in The Gambia, 2) examine the nature (i.e., prevalence of turn taking) and amount (i.e., adult and child vocalizations) of conversation that infants are exposed to from 12 to 24 months of age and 3) investigate the link between caregiver diversity and child language outcomes, examining the mediating role of contingent turn taking.

METHOD

We obtained naturalistic seven-hour-long LENA recordings at 12, 18 and 24 months of age from a cohort of N = 204 infants from Mandinka speaking households in The Gambia and N = 61 infants in the UK. We examined developmental changes and site differences in LENA counts of adult word counts (AWC), contingent turn taking (CTT) and child vocalizations (CVC). In the larger and more heterogenous Gambian sample, we also investigated caregiver predictors of turn taking frequency. We hereby examined the number of caregivers present over the recording day and the consistency of caregivers across two subsequent days per age point. We controlled for children's cognitive development via the Mullen Scales of Early Learning (MSEL).

RESULTS

Our LENA validation showed high internal consistency between the human coders and automated LENA outputs (Cronbach's alpha's all >.8). All LENA counts were higher in the UK compared to the Gambian cohort. In The Gambia, controlling for overall neurodevelopment via the MSEL, CTT at 12 and 18 months predicted CVC at 18 and 24 months. Caregiver consistency was associated with CTT counts at 18 and 24 months. The number of caregivers and CTT counts showed an inverted u-shape relationship at 18 and 24 months, with an intermediate number of caregivers being associated with the highest CTT frequencies. Mediation analyses showed a partial mediation by number of caregivers and CTT and 24-month CVC.

DISCUSSION

The LENA provided reliable estimates for the Mandinka language in the home recording context. We showed that turn taking is associated with subsequent child vocalizations and explored contextual caregiving factors contributing to turn taking in the Gambian cohort.

Assessing Pain Research: A Narrative Review of Emerging Pain Methods, Their Technosocial Implications, and Opportunities for Multidisciplinary Approaches

Pain research traverses many disciplines and methodologies. Yet, despite our understanding and field-wide acceptance of the multifactorial essence of pain as a sensory perception, emotional experience, and biopsychosocial condition, pain scientists and practitioners often remain siloed within their domain expertise and associated techniques. The context in which the field finds itself today-with increasing reliance on digital technologies, an on-going pandemic, and continued disparities in pain care-requires new collaborations and different approaches to measuring pain. Here, we review the state-of-the-art in human pain research, summarizing emerging practices and cutting-edge techniques across multiple methods and technologies. For each, we outline foreseeable technosocial considerations, reflecting on implications for standards of care, pain management, research, and societal impact. Through overviewing alternative data sources and varied ways of measuring pain and by reflecting on the concerns, limitations, and challenges facing the field, we hope to create critical dialogues, inspire more collaborations, and foster new ideas for future pain research methods.

Using big data from long-form recordings to study development and optimize societal impact

Big data are everywhere. In this chapter, we focus on one source: long-form, child-centered recordings collected using wearable technologies. Because these recordings are simultaneously unobtrusive and encompassing, they may be a breakthrough technology for clinicians and researchers from several diverse fields. We demonstrate this possibility by outlining three applications for the recordings-clinical treatment, large-scale interventions, and language documentation-where we see the greatest potential. We argue that incorporating these recordings into basic and applied research will result in more equitable treatment of patients, more reliable measurements of the effects of interventions on real-world behavior, and deeper scientific insights with less observational bias. We conclude by outlining a proposal for a semistructured online platform where vast numbers of long-form recordings could be hosted and more representative, less biased algorithms could be trained.

Daylong Mobile Audio Recordings Reveal Multitimescale Dynamics in Infants' Vocal Productions and Auditory Experiences

The sounds of human infancy-baby babbling, adult talking, lullaby singing, and more-fluctuate over time. Infant-friendly wearable audio recorders can now capture very large quantities of these sounds throughout infants' everyday lives at home. Here, we review recent discoveries about how infants' soundscapes are organized over the course of a day based on analyses designed to detect patterns at multiple timescales. Analyses of infants' day-long audio have revealed that everyday vocalizations are clustered hierarchically in time, vocal explorations are consistent with foraging dynamics, and musical tunes are distributed such that some are much more available than others. This approach focusing on the multi-scale distributions of sounds heard and produced by infants provides new, fundamental insights on human communication development from a complex systems perspective.

Learning Through Processing: Toward an Integrated Approach to Early Word Learning

Children's linguistic knowledge and the learning mechanisms by which they acquire it grow substantially in infancy and toddlerhood, yet theories of word learning largely fail to incorporate these shifts. Moreover, researchers' often-siloed focus on either familiar word recognition or novel word learning limits the critical consideration of how these two relate. As a step toward a mechanistic theory of language acquisition, we present a framework of "learning through processing" and relate it to the prevailing methods used to assess children's early knowledge of words. Incorporating recent empirical work, we posit a specific, testable timeline of qualitative changes in the learning process in this interval. We conclude with several challenges and avenues for building a comprehensive theory of early word learning: better characterization of the input, reconciling results across approaches, and treating lexical knowledge in the nascent grammar with sufficient sophistication to ensure generalizability across languages and development.

Quantifying Everyday Ecologies: Principles for Manual Annotation of Many Hours of Infants' Lives

Everyday experiences are the experiences available to shape developmental change. Remarkable advances in devices used to record infants' and toddlers' everyday experiences, as well as in repositories to aggregate and share such recordings across teams of theorists, have yielded a potential gold mine of insights to spur next-generation theories of experience-dependent change. Making full use of these advances, however, currently requires manual annotation. Manually annotating many hours of everyday life is a dedicated pursuit requiring significant time and resources, and in many domains is an endeavor currently lacking foundational facts to guide potentially consequential implementation decisions. These realities make manual annotation a frequent barrier to discoveries, as theorists instead opt for narrower scoped activities. Here, we provide theorists with a framework for manually annotating many hours of everyday life designed to reduce both theoretical and practical overwhelm. We share insights based on our team's recent adventures in the previously uncharted territory of everyday music. We identify principles, and share implementation examples and tools, to help theorists achieve scalable solutions to challenges that are especially fierce when annotating extended timescales. These principles for quantifying everyday ecologies will help theorists collectively maximize return on investment in databases of everyday recordings and will enable a broad community of scholars—across institutions, skillsets, experiences, and working environments—to make discoveries about the experiences upon which development may depend.

Describing Vocalizations in Young Children: A Big Data Approach Through Citizen Science Annotation

Purpose Recording young children's vocalizations through wearables is a promising method to assess language development. However, accurately and rapidly annotating these files remains challenging. Online crowdsourcing with the collaboration of citizen scientists could be a feasible solution. In this article, we assess the extent to which citizen scientists' annotations align with those gathered in the lab for recordings collected from young children. Method Segments identified by Language ENvironment Analysis as produced by the key child were extracted from one daylong recording for each of 20 participants: 10 low-risk control children and 10 children diagnosed with Angelman syndrome, a neurogenetic syndrome characterized by severe language impairments. Speech samples were annotated by trained annotators in the laboratory as well as by citizen scientists on Zooniverse. All annotators assigned one of five labels to each sample: Canonical, Noncanonical, Crying, Laughing, and Junk. This allowed the derivation of two child-level vocalization metrics: the Linguistic Proportion and the Canonical Proportion. Results At the segment level, Zooniverse classifications had moderate precision and recall. More importantly, the Linguistic Proportion and the Canonical Proportion derived from Zooniverse annotations were highly correlated with those derived from laboratory annotations. Conclusions Annotations obtained through a citizen science platform can help us overcome challenges posed by the process of annotating daylong speech recordings. Particularly when used in composites or derived metrics, such annotations can be used to investigate early markers of language delays.

Early language experience in a Papuan community

The rate at which young children are directly spoken to varies due to many factors, including (a) caregiver ideas about children as conversational partners and (b) the organization of everyday life. Prior work suggests cross-cultural variation in rates of child-directed speech is due to the former factor, but has been fraught with confounds in comparing postindustrial and subsistence farming communities. We investigate the daylong language environments of children (0;0-3;0) on Rossel Island, Papua New Guinea, a small-scale traditional community where prior ethnographic study demonstrated contingency-seeking child interaction styles. In fact, children were infrequently directly addressed and linguistic input rate was primarily affected by situational factors, though children's vocalization maturity showed no developmental delay. We compare the input characteristics between this community and a Tseltal Mayan one in which near-parallel methods produced comparable results, then briefly discuss the models and mechanisms for learning best supported by our findings.

ALICE: An open-source tool for automatic measurement of phoneme, syllable, and word counts from child-centered daylong recordings

Recordings captured by wearable microphones are a standard method for investigating young children's language environments. A key measure to quantify from such data is the amount of speech present in children's home environments. To this end, the LENA recorder and software-a popular system for measuring linguistic input-estimates the number of adult words that children may hear over the course of a recording. However, word count estimation is challenging to do in a language- independent manner; the relationship between observable acoustic patterns and language-specific lexical entities is far from uniform across human languages. In this paper, we ask whether some alternative linguistic units, namely phone(me)s or syllables, could be measured instead of, or in parallel with, words in order to achieve improved cross-linguistic applicability and comparability of an automated system for measuring child language input. We discuss the advantages and disadvantages of measuring different units from theoretical and technical points of view. We also investigate the practical applicability of measuring such units using a novel system called Automatic LInguistic unit Count Estimator (ALICE) together with audio from seven child-centered daylong audio corpora from diverse cultural and linguistic environments. We show that language-independent measurement of phoneme counts is somewhat more accurate than syllables or words, but all three are highly correlated with human annotations on the same data. We share an open-source implementation of ALICE for use by the language research community, enabling automatic phoneme, syllable, and word count estimation from child-centered audio recordings.

Advances in microanalysis: Magnifying the social microscope on mother-infant interactions

Microanalysis is a method for recording and coding interactional behavior. It has been often compared to a social microscope, for its power in detailing the second-by-second dynamics of social interaction. Microanalysis has deep multidisciplinary foundations, that privilege the description of interactions as they naturally occur, with the purpose of understanding the relations between multiple and simultaneous streams of behaviors. In developmental science, microanalysis has uncovered structural and temporal elements in mother-infant interactions, improving our understanding of the effects of mother-infant interpersonal adaptation in the infant's cognitive and social-emotional development. Detailed manual coding is time intensive and resource demanding, imposing restrictions to sample size, and the ability to analyze multiple behavioral modalities. Moreover, recent increases in the density of multivariate data require different tools. We review present-day techniques that tackle those challenges: (1) sensing techniques for motion tracking and physiological recording; (2) exploratory techniques for detecting patterns from high-density data; and (3) inferential and modeling techniques for understanding contingencies between interactional time series. Two illustrations, from recent developmental research, reveal the power of bringing new lenses to our social microscope: (1) egocentric vision, the use of head mounted cameras and eye-trackers in capturing the infant's first-person perspective of a social exchange; and (2) daily activity sensing, wearable multimodal sensing that brought mother-infant interaction research to the environments where it naturally unfolds.

Practice and experience predict coarticulation in child speech

Much research in child speech development suggests that young children coarticulate more than adults. There are multiple, not mutually-exclusive, explanations for this pattern. For example, children may coarticulate more because they are limited by immature motor control. Or they may coarticulate more if they initially represent phonological segments in larger, more holistic units such as syllables or feet. We tested the importance of several different explanations for coarticulation in child speech by evaluating how four-year-olds' language experience, speech practice, and speech planning predicted their coarticulation between adjacent segments in real words and paired nonwords. Children with larger vocabularies coarticulated less, especially in real words, though there were no reliable coarticulatory differences between real words and nonwords after controlling for word duration. Children who vocalized more throughout a daylong audio recording also coarticulated less. Quantity of child vocalizations was more predictive of the degree of children's coarticulation than a measure of receptive language experience, adult word count. Overall, these results suggest strong roles for children's phonological representations and speech practice, as well as their immature fine motor control, for coarticulatory development.

Modeling the Influence of Language Input Statistics on Children's Speech Production

We trained a computational model (the Chunk-Based Learner; CBL) on a longitudinal corpus of child-caregiver interactions in English to test whether one proposed statistical learning mechanism-backward transitional probability-is able to predict children's speech productions with stable accuracy throughout the first few years of development. We predicted that the model less accurately reconstructs children's speech productions as they grow older because children gradually begin to generate speech using abstracted forms rather than specific "chunks" from their speech environment. To test this idea, we trained the model on both recently encountered and cumulative speech input from a longitudinal child language corpus. We then assessed whether the model could accurately reconstruct children's speech. Controlling for utterance length and the presence of duplicate chunks, we found no evidence that the CBL becomes less accurate in its ability to reconstruct children's speech with age.

Longform recordings of everyday life: Ethics for best practices

Recent advances in large-scale data storage and processing offer unprecedented opportunities for behavioral scientists to collect and analyze naturalistic data, including from underrepresented groups. Audio data, particularly real-world audio recordings, are of particular interest to behavioral scientists because they provide high-fidelity access to subtle aspects of daily life and social interactions. However, these methodological advances pose novel risks to research participants and communities. In this article, we outline the benefits and challenges associated with collecting, analyzing, and sharing multi-hour audio recording data. Guided by the principles of autonomy, privacy, beneficence, and justice, we propose a set of ethical guidelines for the use of longform audio recordings in behavioral research. This article is also accompanied by an Open Science Framework Ethics Repository that includes informed consent resources such as frequent participant concerns and sample consent forms.

The role of caregiver speech in supporting language development in infants and toddlers with autism spectrum disorder

Parents play an essential role in supporting child development by providing a safe home, proper nutrition, and rich educational opportunities. In this article we focus on the role of caregiver speech in supporting development of young children with autism spectrum disorder (ASD). We review studies from typically developing children and children with autism showing that rich and responsive caregiver speech supports language development. Autism intervention studies that target caregiver speech are reviewed as are recent scientific advances from studies of typical development. The strengths and weakness of different techniques for collecting language data from caregivers and children are reviewed, and natural language samples are recommended as best practice for language research in autism. We conclude that caregivers play a powerful role in shaping their children's development and encourage researchers to adapt parent-mediated intervention studies to acknowledge individual differences in parents by using a personalized medicine approach.