Navigating through the experienced environment: Insights from mobile eye tracking

Strategies for enhancing automatic fixation detection in head-mounted eye tracking

Moving through a dynamic world, humans need to intermittently stabilize gaze targets on their retina to process visual information. Overt attention being thus split into discrete intervals, the automatic detection of such fixation events is paramount to downstream analysis in many eye-tracking studies. Standard algorithms tackle this challenge in the limiting case of little to no head motion. In this static scenario, which is approximately realized for most remote eye-tracking systems, it amounts to detecting periods of relative eye stillness. In contrast, head-mounted eye trackers allow for experiments with subjects moving naturally in everyday environments. Detecting fixations in these dynamic scenarios is more challenging, since gaze-stabilizing eye movements need to be reliably distinguished from non-fixational gaze shifts. Here, we propose several strategies for enhancing existing algorithms developed for fixation detection in the static case to allow for robust fixation detection in dynamic real-world scenarios recorded with head-mounted eye trackers. Specifically, we consider (i) an optic-flow-based compensation stage explicitly accounting for stabilizing eye movements during head motion, (ii) an adaptive adjustment of algorithm sensitivity according to head-motion intensity, and (iii) a coherent tuning of all algorithm parameters. Introducing a new hand-labeled dataset, recorded with the Pupil Invisible glasses by Pupil Labs, we investigate their individual contributions. The dataset comprises both static and dynamic scenarios and is made publicly available. We show that a combination of all proposed strategies improves standard thresholding algorithms and outperforms previous approaches to fixation detection in head-mounted eye tracking.

Now it's your turn!: Eye blink rate in a Jenga task modulated by interaction of task wait times, effortful control, and internalizing behaviors

Dopamine is a versatile neurotransmitter with implications in many domains, including anxiety and effortful control. Where high levels of effortful control are often regarded as adaptive, other work suggests that high levels of effortful control may be a risk factor for anxiety. Dopamine signaling may be key in understanding these relations. Eye blink rate is a non-invasive proxy metric of midbrain dopamine activity. However, much work with eye blink rate has been constrained to screen-based tasks which lack in ecological validity. We tested whether changes in eye blink rate during a naturalistic effortful control task differ as a function of parent-reported effortful control and internalizing behaviors. Children played a Jenga-like game with an experimenter, but for each trial the experimenter took an increasingly long time to take their turn. Blinks-per-second were computed during each wait period. Multilevel modeling examined the relation between duration of wait period, effortful control, and internalizing behaviors on eye blink rate. We found a significant 3-way interaction between effortful control, internalizing behaviors, and duration of the wait period. Probing this interaction revealed that for children with low reported internalizing behaviors (-1 SD) and high reported effortful control (+1 SD), eye blink rate significantly decreased as they waited longer to take their turn. These findings index task-related changes in midbrain dopamine activity in relation to naturalistic task demands, and that these changes may vary as a function of individual differences in effortful control and internalizing behaviors. We discuss possible top-down mechanisms that may underlie these differences.

Infant embodied attention in context: Feasibility of home-based head-mounted eye tracking in early infancy

Social communication emerges from dynamic, embodied social interactions during which infants coordinate attention to caregivers and objects. Yet many studies of infant attention are constrained to a laboratory setting, neglecting how attention is nested within social contexts where caregivers dynamically scaffold infant behavior in real time. This study evaluates the feasibility and acceptability of the novel use of head-mounted eye tracking (HMET) in the home with N = 40 infants aged 4 and 8 months who are typically developing and at an elevated genetic liability for autism spectrum disorder (ASD). Results suggest that HMET with young infants with limited independent motor abilities and at an elevated likelihood for atypical development is highly feasible and deemed acceptable by caregivers. Feasibility and acceptability did not differ by age or ASD likelihood. Data quality was also acceptable, albeit with younger infants showing slightly lower accuracy, allowing for preliminary analysis of developmental trends in infant gaze behavior. This study provides new evidence for the feasibility of using in-home HMET with young infants during a critical developmental period when more complex interactions with the environment and social partners are emerging. Future research can apply this technology to illuminate atypical developmental trajectories of embodied social attention in infancy.

Early social communication through music: State of the art and future perspectives

A growing body of research shows that the universal capacity for music perception and production emerges early in development. Possibly building on this predisposition, caregivers around the world often communicate with infants using songs or speech entailing song-like characteristics. This suggests that music might be one of the earliest developing and most accessible forms of interpersonal communication, providing a platform for studying early communicative behavior. However, little research has examined music in truly communicative contexts. The current work aims to facilitate the development of experimental approaches that rely on dynamic and naturalistic social interactions. We first review two longstanding lines of research that examine musical interactions by focusing either on the caregiver or the infant. These include defining the acoustic and non-acoustic features that characterize infant-directed (ID) music, as well as behavioral and neurophysiological research examining infants' processing of musical timing and pitch. Next, we review recent studies looking at early musical interactions holistically. This research focuses on how caregivers and infants interact using music to achieve co-regulation, mutual engagement, and increase affiliation and prosocial behavior. We conclude by discussing methodological, technological, and analytical advances that might empower a comprehensive study of musical communication in early childhood.

Relations between social attention, expressed positive affect and behavioral inhibition during play

Flexible social attention, including visually attending to social interaction partners, coupled with positive affect may facilitate adaptive social functioning. However, most research assessing social attention relies on static computer-based paradigms, overlooking the dynamics of social interactions and limiting understanding of individual differences in the deployment of naturalistic attention. The current study used mobile eye-tracking to examine relations between social attention, expressed affect, and behavioral inhibition during naturalistic play in young children. Children (N = 28, Mage = 6.12, 46.4% girls, 92.9% White) participated in a 5-min free play with a novel age- and sex-matched peer while mobile eye-tracking data were collected. Interactions were coded for social attention and expressed affect and modeled second-by-second, generating 4,399 observations. Children spent more time dwelling on toys than on peers or anywhere else in the room. Further analyses demonstrated children were almost twice as likely to gaze at their peer when simultaneously self-expressing positive affect. Additionally, children were more than twice as likely and more than three times as likely to self-express positive affect when dwelling on peer or in the presence of peer-expressed positive affect, respectively. Behavioral inhibition was not significantly related to social attention. However, children higher in behavioral inhibition were less likely to self-express positive affect in the presence of peer-expressed positive affect. The current results provide a snapshot of relations between social attention, expressed affect and individual differences during play and provide guidance for future work assessing the roles of social attention and positive affect in facilitating positive social interactions. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Sensory Appeal and Routines Beat Health Messages and Visibility Enhancements: Mixed-Methods Analysis of a Choice-Architecture Intervention in a Workplace Cafeteria

Easier recognition and enhanced visibility of healthy options supposedly increase healthy choices, but real-world evidence remains scarce. Addressing this knowledge gap, we promoted nutritionally favourable foods in a workplace cafeteria with three choice-architectural strategies-priming posters, point-of-choice nutrition labels, and improved product placement-and assessed their effects on visual attention, food choices, and food consumption. Additionally, we developed a method for analysing real-world eye-tracking data. The study followed a pretest-posttest design whereby control and intervention condition lasted five days each. We monitored visual attention (i.e., total number and duration of fixations) and food choices with eye tracking, interviewed customers about perceived influences on food choices, and measured cafeteria-level food consumption (g). Individual-level data represents 22 control and 19 intervention participants recruited at the cafeteria entrance. Cafeteria-level data represents food consumption during the trial (556/589 meals sold). Results indicated that the posters and labels captured participants' visual attention (~13% of fixations on defined areas of interest before food choices), but the intervention had insignificant effects on visual attention to foods, on food choices, and on food consumption. Interviews revealed 17 perceived influences on food choices, the most common being sensory appeal, healthiness, and familiarity. To conclude, the intervention appeared capable of attracting visual attention, yet ineffective in increasing healthier eating. The developed method enabled a rigorous analysis of visual attention and food choices in a natural choice setting. We discuss ways to boost the impact of the intervention on behaviour, considering target groups' motives. The work contributes with a unique, mixed-methods approach and a real-world setting that enabled a multi-dimensional effects evaluation with high external validity.

Eye contact avoidance in crowds: A large wearable eye-tracking study

Eye contact is essential for human interactions. We investigated whether humans are able to avoid eye contact while navigating crowds. At a science festival, we fitted 62 participants with a wearable eye tracker and instructed them to walk a route. Half of the participants were further instructed to avoid eye contact. We report that humans can flexibly allocate their gaze while navigating crowds and avoid eye contact primarily by orienting their head and eyes towards the floor. We discuss implications for crowd navigation and gaze behavior. In addition, we address a number of issues encountered in such field studies with regard to data quality, control of the environment, and participant adherence to instructions. We stress that methodological innovation and scientific progress are strongly interrelated.

Do you see what I mean?: Using mobile eye tracking to capture parent-child dynamics in the context of anxiety risk

Temperamental behavioral inhibition (BI) is a robust endophenotype for anxiety characterized by increased sensitivity to novelty. Controlling parenting can reinforce children's wariness by rewarding signs of distress. Fine-grained, dynamic measures are needed to better understand both how children perceive their parent's behaviors and the mechanisms supporting evident relations between parenting and socioemotional functioning. The current study examined dyadic attractor patterns (average mean durations) with state space grids, using children's attention patterns (captured via mobile eye tracking) and parental behavior (positive reinforcement, teaching, directives, intrusion), as functions of child BI and parent anxiety. Forty 5- to 7-year-old children and their primary caregivers completed a set of challenging puzzles, during which the child wore a head-mounted eye tracker. Child BI was positively correlated with proportion of parent's time spent teaching. Child age was negatively related, and parent anxiety level was positively related, to parent-focused/controlling parenting attractor strength. There was a significant interaction between parent anxiety level and child age predicting parent-focused/controlling parenting attractor strength. This study is a first step to examining the co-occurrence of parenting behavior and child attention in the context of child BI and parental anxiety levels.

The call for ecological validity is right but missing perceptual idiosyncrasies is wrong

Although psychology has long professed that perception predicts action, the strength of the evidence supporting the statement depends on the ecological validity of the technologies and paradigms used, particularly those that track eye movements, supporting Cesario's argument. While right to call for ecological validity, Cesario's model fails to account for individual differences in visual experience perceivers have when presented with the same stimulus.

Beyond screen time: Using head-mounted eye tracking to study natural behavior

Head-mounted eye tracking is a new method that allows researchers to catch a glimpse of what infants and children see during naturalistic activities. In this chapter, we review how mobile, wearable eye trackers improve the construct validity of important developmental constructs, such as visual object experiences and social attention, in ways that would be impossible using screen-based eye tracking. Head-mounted eye tracking improves ecological validity by allowing researchers to present more realistic and complex visual scenes, create more interactive experimental situations, and examine how the body influences what infants and children see. As with any new method, there are difficulties to overcome. Accordingly, we identify what aspects of head-mounted eye-tracking study design affect the measurement quality, interpretability of the results, and efficiency of gathering data. Moreover, we provide a summary of best practices aimed at allowing researchers to make well-informed decisions about whether and how to apply head-mounted eye tracking to their own research questions.

Parent-to-Child Anxiety Transmission Through Dyadic Social Dynamics: A Dynamic Developmental Model

The intergenerational transmission of psychopathology is one of the strongest known risk factors for childhood disorder and may be a malleable target for prevention and intervention. Anxious parents have distinct parenting profiles that impact socioemotional development, and these parenting effects may result in broad alterations to the biological and cognitive functioning of their children. Better understanding the functional mechanisms by which parental risk is passed on to children can provide (1) novel markers of risk for socioemotional difficulties, (2) specific targets for intervention, and (3) behavioral and biological indices of treatment response. We propose a developmental model in which dyadic social dynamics serve as a key conduit in parent-to-child transmission of anxiety. Dyadic social dynamics capture the moment-to-moment interactions between parent and child that occur on a daily basis. In shaping the developmental trajectory from familial risk to actual symptoms, dyadic processes act on mechanisms of risk that are evident prior to, and in the absence of, any eventual disorder onset. First, we discuss dyadic synchrony or the moment-to-moment coordination between parent and child within different levels of analysis, including neural, autonomic, behavioral, and emotional processes. Second, we discuss how overt emotion modeling of distress is observed and internalized by children and later reflected in their own behavior. Thus, unlike synchrony, this is a more sequential process that cuts across levels of analysis. We also discuss maladaptive cognitive and affective processing that is often evident with increases in child anxiety symptoms. Finally, we discuss additional moderators (e.g., parent sex, child fearful temperament) that may impact dyadic processes. Our model is proposed as a conceptual framework for testing hypotheses regarding dynamic processes that may ultimately guide novel treatment approaches aimed at intervening on dyadically linked biobehavioral mechanisms before symptom onset.

Mobile Eye Tracking Captures Changes in Attention Over Time During a Naturalistic Threat Paradigm in Behaviorally Inhibited Children

Attentional biases to and away from threat are considered hallmarks of temperamental Behavioral Inhibition (BI), which is a documented risk factor for social anxiety disorder. However, most research on affective attentional biases has traditionally been constrained to computer screens, where stimuli often lack ecological validity. Moreover, prior research predominantly focuses on momentary presentations of stimuli, rather than examining how attention may change over the course of prolonged exposure to salient people and objects. Here, in a sample of children oversampled for BI, we used mobile eye-tracking to examine attention to an experimenter wearing a "scary" or novel gorilla mask, as well as attention to the experimenter after mask removal as a recovery from exposure. Conditional growth curve modeling was used to examine how level of BI related to attentional trajectories over the course of the exposure. We found a main effect of BI in the initial exposure to the mask, with a positive association between level of BI and proportion of gaze allocated to the stranger's masked face over time. Additionally, there was a main effect of BI on proportion of gaze allocated to the stranger's face plus their mask during the recovery period when the mask was removed.

Profiles of Naturalistic Attentional Trajectories Associated with Internalizing Behaviors in School-Age Children: A Mobile Eye Tracking Study

The temperament profile Behavioral Inhibition (BI) is a strong predictor of internalizing behaviors in childhood. Patterns of attention towards or away from threat are a commonality of both BI and internalizing behaviors. Attention biases are traditionally measured with computer tasks presenting affective stimuli, which can lack ecological validity. Recent studies suggest that naturalistic visual attention need not mirror findings from computer tasks, and, more specifically, children high in BI may attend less to threats in naturalistic tasks. Here, we characterized latent trajectories of naturalistic visual attention over time to a female stranger, measured with mobile eye tracking, among kindergarteners oversampled for BI. Group-based trajectory modeling (GBTM) revealed two latent trajectories: 1) high initial orienting to the stranger, gradual decay, and recovery, and 2) low initial orienting and continued avoidance. Higher probability of membership to the "avoidant" group was linked to greater report of internalizing behaviors. We demonstrate the efficacy of mobile eye tracking in quantifying naturalistic patterns of visual attention to social novelty, as well as the importance of naturalistic measures of attention in characterizing socioemotional risk factors.

Using mobile eye-tracking technology to examine adolescent daughters' attention to maternal affect during a conflict discussion

Attention to socio-emotional stimuli (i.e., affect-biased attention) is an integral component of emotion regulation and human communication. Given the strong link between maternal affect and adolescent behavior, maternal affect may be a critical influence on adolescent affect-biased attention during mother-child interaction. However, prior methodological constraints have precluded fine-grained examinations of factors such as maternal affect on adolescent attention during real-world social interaction. Therefore, this pilot study capitalized on previously validated technological advances by using mobile eye-tracking and facial affect coding software to quantify the influence of maternal affect on adolescents' attention to the mother during a conflict discussion. Results from 7,500 to 9,000 time points sampled for each mother-daughter dyad (n = 28) indicated that both negative and positive maternal affect, relative to neutral, elicited more adolescent attentional avoidance of the mother (ORs = 2.68-9.20), suggesting that typically developing adolescents may seek to avoid focusing on maternal affect of either valence during a conflict discussion. By examining the moment-to-moment association between in vivo displays of maternal affect and subsequent adolescent attention toward the mother's face, these results provide preliminary evidence that maternal affect moderates adolescent attention. Our findings are consistent with cross-species approach-avoidance models suggesting that offspring respond to affectively charged conversations with greater behavioral avoidance or deference.

An interactionist perspective on the development of coordinated social attention

Infants' ability to coordinate their attention with other people develops profoundly across the first year of life. Mainly based on experimental research focusing on infants' behavior under highly controlled conditions, developmental milestones were identified and explained in the past by prominent theories in terms of the onset of specific cognitive skills. In contrast to this approach, recent longitudinal research challenges this perspective with findings suggesting that social attention develops continuously with a gradual refinement of skills. Informed by these findings, we argue for an interactionist and dynamical systems view that bases observable advances in infant social attention skills on increasingly fine-tuned mutual adjustments in the caregiver-infant dyad, resulting in gradually improving mutual prediction. We present evidence for this view from recent studies leveraging new technologies which afford the opportunity to dynamically track social interactions in real-time. These new technically-sophisticated studies offer unprecedented insights into the dynamic processes of infant-caregiver social attention. It is now possible to track in much greater detail fluctuations over time with regard to object-directed attention as well as social attention and how these processes relate to one another. Encouraged by these initial results and new insights from this interactionist developmental social neuroscience approach, we conclude with a "call to action" in which we advocate for more ecologically valid paradigms for studying social attention as a dynamic and bi-directional process.

Vigilant or avoidant? Children's temperamental shyness, patterns of gaze, and physiology during social threat

Temperamental shyness is characterized by fear, wariness, and the perception of threat in response to social novelty. Previous work has been inconsistent regarding attentional patterns to social threat among shy children, with evidence for both avoidance and vigilance. We examined relations between children's shyness and gaze aversion during the approach of a stranger (i.e., a context of social novelty), and tested whether these patterns of gaze moderated relations between shyness and autonomic reactivity and recovery. Participants included 152 typically-developing children (M_age  = 7.82 years, SD = 0.44 years) who had their respiratory sinus arrhythmia (RSA) recorded during baseline, social novelty, and recovery. Children's shyness correlated with increases in self-reported nervousness from baseline to social novelty, providing support for perceived threat. Results revealed that children's proportion of gaze aversion from social novelty was related to shyness in a U-shape pattern such that both low levels of gaze aversion (i.e., attentional vigilance) and high levels of gaze aversion (i.e., attentional avoidance) were related to higher levels of shyness. Further, we found that children's shyness was directly related to decreases in RSA from baseline to social novelty, whereas quadratic gaze to social novelty moderated the relation between shyness and RSA recovery. Specifically, shyness was related to greater RSA recovery among children who exhibited attentional vigilance during the novel social interaction. Our findings provide support for both avoidance of, and vigilance to, social threat among different shy children, and these gaze strategies may be differentially related to physiological regulation during novel social encounters.

Variable- and person-centered approaches to affect-biased attention in infancy reveal unique relations with infant negative affect and maternal anxiety

Affect-biased attention is an automatic process that prioritizes emotionally or motivationally salient stimuli. Several models of affect-biased attention and its development suggest that it comprises an individual's ability to both engage with and disengage from emotional stimuli. Researchers typically rely on singular tasks to measure affect-biased attention, which may lead to inconsistent results across studies. Here we examined affect-biased attention across three tasks in a unique sample of 193 infants, using both variable-centered (factor analysis; FA) and person-centered (latent profile analysis; LPA) approaches. Using exploratory FA, we found evidence for two factors of affect-biased attention: an Engagement factor and a Disengagement factor, where greater maternal anxiety was related to less engagement with faces. Using LPA, we found two groups of infants with different patterns of affect-biased attention: a Vigilant group and an Avoidant group. A significant interaction noted that infants higher in negative affect who also had more anxious mothers were most likely to be in the Vigilant group. Overall, these results suggest that both FA and LPA are viable approaches for studying distinct questions related to the development of affect-biased attention, and set the stage for future longitudinal work examining the role of infant negative affect and maternal anxiety in the emergence of affect-biased attention.

Eye tracking in human interaction: Possibilities and limitations

There is a long history of interest in looking behavior during human interaction. With the advance of (wearable) video-based eye trackers, it has become possible to measure gaze during many different interactions. We outline the different types of eye-tracking setups that currently exist to investigate gaze during interaction. The setups differ mainly with regard to the nature of the eye-tracking signal (head- or world-centered) and the freedom of movement allowed for the participants. These features place constraints on the research questions that can be answered about human interaction. We end with a decision tree to help researchers judge the appropriateness of specific setups.

Action prediction during real-time parent-infant interactions

Social interactions provide a crucial context for early learning and cognitive development during infancy. Action prediction-the ability to anticipate an observed action-facilitates successful, coordinated interaction and is an important social-cognitive skill in early development. However, current knowledge about infant action prediction comes largely from screen-based laboratory tasks. We know little about what infants' action prediction skills look like during real-time, free-flowing interactions with a social partner. In the current study, we used head-mounted eyetracking to quantify 9-month-old infants' visual anticipations of their parents' actions during free-flowing parent-child play. Our findings reveal that infants do anticipate their parents' actions during dynamic interactions at rates significantly higher than would be expected by chance. In addition, the frequency with which they do so is associated with child-led joint attention and hand-eye coordination. These findings are the first to reveal infants' action prediction behaviors in a more naturalistic context than prior screen-based studies, and they support the idea that action prediction is inherently linked to motor development and plays an important role in infants' social-cognitive development. A video abstract of this article can be viewed at https://www.youtube.com/watch?v=9HrmcicfiqE.