Inequality-Constrained 3D Morphable Face Model Fitting

3D morphable model (3DMM) fitting on 2D data is traditionally done via unconstrained optimization with regularization terms to ensure that the result is a plausible face shape and is consistent with a set of 2D landmarks. This paper presents inequality-constrained 3DMM fitting as the first alternative to regularization in optimization-based 3DMM fitting. Inequality constraints on the 3DMM's shape coefficients ensure face-like shapes without modifying the objective function for smoothness, thus allowing for more flexibility to capture person-specific shape details. Moreover, inequality constraints on landmarks increase robustness in a way that does not require per-image tuning. We show that the proposed method stands out with its ability to estimate person-specific face shapes by jointly fitting a 3DMM to multiple frames of a person. Further, when used with a robust objective function, namely gradient correlation, the method can work "in-the-wild" even with a 3DMM constructed from controlled data. Lastly, we show how to use the log-barrier method to efficiently implement the method. To our knowledge, we present the first 3DMM fitting framework that requires no learning yet is accurate, robust, and efficient. The absence of learning enables a generic solution that allows flexibility in the input image size, interchangeable morphable models, and incorporation of camera matrix.

Measuring Dyspraxia in Autism using a Five-Minute Praxis Exam

Background

Difficulties with praxis, the ability to perform learned skilled movements, have been robustly demonstrated in autism spectrum disorder (autism). However, praxis assessment is not routinely included in autism characterization batteries, in part because it is traditionally time consuming to administer and score. We test whether dyspraxia in autism can be captured with a brief measure.

Method

Youth with autism (n = 41) and matched typically developing controls (n = 32), aged 8 to 16 years, completed a 5-minute praxis battery. The 19-item battery included four subtests: gesture to command, tool use, familiar imitation, and meaningless imitation. Video recordings were coded for error types and compared to participant characterization variables.

Results

Consistent with research using a lengthy battery, autistic youth made more errors overall, with a large effect size. Groups demonstrated similar distributions of error types, suggesting that dyspraxia in autism is not limited to a particular error form. In the autism group, praxis was associated with adaptive functioning, but not autism traits.

Conclusions

A shortened battery is sufficiently sensitive to praxis differences between autistic and typically developing youth, increasing the feasibility of including praxis within clinical assessments or larger research batteries aimed at testing relationships with downstream skills.

Predicting Autism from Head Movement Patterns during Naturalistic Social Interactions

Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized in part by difficulties in verbal and nonverbal social communication. Evidence indicates that autistic people, compared to neurotypical peers, exhibit differences in head movements, a key form of nonverbal communication. Despite the crucial role of head movements in social communication, research on this nonverbal cue is relatively scarce compared to other forms of nonverbal communication, such as facial expressions and gestures. There is a need for scalable, reliable, and accurate instruments for measuring head movements directly within the context of social interactions. In this study, we used computer vision and machine learning to examine the head movement patterns of neurotypical and autistic individuals during naturalistic, face-to-face conversations, at both the individual (monadic) and interpersonal (dyadic) levels. Our model predicts diagnostic status using dyadic head movement data with an accuracy of 80%, highlighting the value of head movement as a marker of social communication. The monadic data pipeline had lower accuracy (69.2%) compared to the dyadic approach, emphasizing the importance of studying back-and-forth social communication cues within a true social context. The proposed classifier is not intended for diagnostic purposes, and future research should replicate our findings in larger, more representative samples.

Comparison of Human Experts and AI in Predicting Autism from Facial Behavior

Advances in computational behavior analysis via artificial intelligence (AI) promise to improve mental healthcare services by providing clinicians with tools to assist diagnosis or measurement of treatment outcomes. This potential has spurred an increasing number of studies in which automated pipelines predict diagnoses of mental health conditions. However, a fundamental question remains unanswered: How do the predictions of the AI algorithms correspond and compare with the predictions of humans? This is a critical question if AI technology is to be used as an assistive tool, because the utility of an AI algorithm would be negligible if it provides little information beyond what clinicians can readily infer. In this paper, we compare the performance of 19 human raters (8 autism experts and 11 non-experts) and that of an AI algorithm in terms of predicting autism diagnosis from short (3-minute) videos of N = 42 participants in a naturalistic conversation. Results show that the AI algorithm achieves an average accuracy of 80.5%, which is comparable to that of clinicians with expertise in autism (83.1%) and clinical research staff without specialized expertise (78.3%). Critically, diagnoses that were inaccurately predicted by most humans (experts and non-experts, alike) were typically correctly predicted by AI. Our results highlight the potential of AI as an assistive tool that can augment clinician diagnostic decision-making.

Head Movement Patterns during Face-to-Face Conversations Vary with Age

Advances in computational behavior analysis have the potential to increase our understanding of behavioral patterns and developmental trajectories in neurotypical individuals, as well as in individuals with mental health conditions marked by motor, social, and emotional difficulties. This study focuses on investigating how head movement patterns during face-to-face conversations vary with age from childhood through adulthood. We rely on computer vision techniques due to their suitability for analysis of social behaviors in naturalistic settings, since video data capture can be unobtrusively embedded within conversations between two social partners. The methods in this work include unsupervised learning for movement pattern clustering, and supervised classification and regression as a function of age. The results demonstrate that 3-minute video recordings of head movements during conversations show patterns that distinguish between participants that are younger vs. older than 12 years with 78% accuracy. Additionally, we extract relevant patterns of head movement upon which the age distinction was determined by our models.

Reconstructing 3D Face of Infants in Social Interactions Using Morphable Models of Non-Infants

3D morphable models (3DMMs) simultaneously reconstruct facial morphology, expression and pose from 2D images, and thus could be an invaluable tool for capturing and characterizing the face and facial behavior in early childhood. However, 3DMM fitting on infants is a largely unexplored problem. All publicly available 3DMMs are developed for adults, and it is unclear if and to what extent they can be used on videos of infants. In this paper, we compare five state-of-the-art 3DMM fitting methods on data from naturalistic infant-caregiver interactions. Results suggest that it is possible to produce consistent and subject-specific reconstructions of 3D shape identity from multiple frames, but not from a single frame. Qualitative evaluation highlights that facial regions with high texture variation, such as eyes, brows and mouth, are captured with higher accuracy compared to the rest of the face. Thus, even though a 3DMM developed for adults has significant limitations when reconstructing the morphology of the entire facial region of infants, applications that involve analysis of facial behavior can be feasible. Our encouraging results, combined with the unique ability of 3DMMs to disentangle two major sources of noise for expression analysis (i.e., identity bias and pose variations), motivate future research on using 3DMMs to measure the facial behavior of infants.

Conversational adaptation in children and teens with autism: Differences in talkativeness across contexts

Successful social communication is complex; it relies on effectively deploying and continuously revising one's behavior to fit the needs of a given conversation, partner, and context. For example, a skilled conversationalist may instinctively become less talkative with a quiet partner and more talkative with a chattier one. Prior research suggests that behavioral flexibility across social contexts can be a particular challenge for individuals with autism spectrum condition (ASC), and that difficulty adapting to the changing needs of a conversation contributes to communicative breakdowns and poor social outcomes. In this study, we examine whether reduced conversational adaptation, as measured by talkativeness, differentiates 48 verbally fluent children and teens with ASC from 50 neurotypical (NT) peers matched on age, intelligence quotient, and sex ratio. Participants completed the Contextual Assessment of Social Skills with two novel conversation partners. The first acted interested in the conversation and talked more (Interested condition), while the second acted bored and talked less (Bored condition). Results revealed that NT participants emulated their conversation partner's behavior by being more talkative in the Interested condition as compared to the Bored condition (z = 9.92, p < 0.001). In contrast, the ASC group did not differentially adapt their behavior to the Bored versus Interested context, instead remaining consistently talkative in both (p = 0.88). The results of this study have implications for understanding social communication and behavioral adaptation in ASC, and may be valuable for clinicians interested in improving conversational competence in verbally fluent individuals with autism. LAY SUMMARY: Social communication-including everyday conversations-can be challenging for individuals on the autism spectrum. In successful conversations, people tend to adjust aspects of their language to be more similar to their partners'. In this study, we found that children and teens with autism did not change their own talkativeness in response to a social partner who was more or less talkative, whereas neurotypical peers did. These findings have clinical implications for improving conversational competence in verbally fluent individuals with autism.

Friend matters: sex differences in social language during autism diagnostic interviews

BACKGROUND

Autistic individuals frequently experience social communication challenges. Girls are diagnosed with autism less often than boys even when their symptoms are equally severe, which may be due to insufficient understanding of the way autism manifests in girls. Differences in the behavioral presentation of autism, including how people talk about social topics, could contribute to these persistent problems with identification. Despite a growing body of research suggesting that autistic girls and boys present distinct symptom profiles in a variety of domains, including social attention, friendships, social motivation, and language, differences in the way that autistic boys and girls communicate verbally are not yet well understood. Closely analyzing boys' and girls' socially-focused language during semi-structured clinical assessments could shed light on potential sex differences in the behavioral presentation of autistic individuals that may prove useful for identifying and effectively supporting autistic girls. Here, we compare social word use in verbally fluent autistic girls and boys during the interview sections of the ADOS-2 Module 3 and measure associations with clinical phenotype.

METHODS

School-aged girls and boys with autism (N = 101, 25 females; aged 6-15) were matched on age, IQ, and parent/clinician ratings of autism symptom severity. Our primary analysis compared the number of social words produced by autistic boys and girls (normalized to account for differences in total word production). Social words are words that make reference to other people, including friends and family.

RESULTS

There was a significant main effect of sex on social word production, such that autistic girls used more social words than autistic boys. To identify the specific types of words driving this effect, additional subcategories of friend and family words were analyzed. There was a significant effect of sex on friend words, with girls using significantly more friend words than boys. However, there was no significant main effect of sex on family words, suggesting that sex differences in social word production may be driven by girls talking more about friends compared to boys, not family. To assess relationships between word use and clinical phenotype, we modeled ADOS-2 Social Affect (SA) scores as a function of social word production. In the overall sample, social word use correlated significantly with ADOS-2 SA scores, indicating that participants who used more social words were rated as less socially impaired by clinicians. However, when examined in each sex separately, this result only held for boys.

LIMITATIONS

This study cannot speak to the ways in which social word use may differ for younger children, adults, or individuals who are not verbally fluent; in addition, there were more autistic boys than girls in our sample, making it difficult to detect small effects.

CONCLUSIONS

Autistic girls used significantly more social words than boys during a diagnostic assessment-despite being matched on age, IQ, and both parent- and clinician-rated autism symptom severity. Sex differences in linguistic markers of social phenotype in autism are especially important in light of the late or missed diagnoses that disproportionately affect autistic girls. Specifically, heightened talk about social topics could complicate autism referral and diagnosis when non-clinician observers expect a male-typical pattern of reduced social focus, which autistic girls may not always exhibit.

Gross motor impairment and its relation to social skills in autism spectrum disorder: A systematic review and two meta-analyses

Gross motor ability is associated with profound differences in how children experience and interact with their social world. A rapidly growing literature on motor development in autism spectrum disorder (ASD) indicates that autistic individuals exhibit impairment in gross motor skills. However, due to substantial heterogeneity across studies, it remains unclear which gross motor skills are impaired in ASD, when and for whom these differences emerge, and whether motor and social impairments are related. The present article addressed these questions by synthesizing research on gross motor skills in ASD in two separate meta-analyses. The first examined gross motor deficits in ASD compared to neurotypical (NT) controls, aggregating data from 114 studies representing 6,423 autistic and 2,941 NT individuals. Results demonstrated a significant overall deficit in gross motor skills in ASD (Hedges' g = -1.04) that was robust to methodological and phenotypic variation and was significant at every level of the tested moderators. However, moderation analyses revealed that this deficit was most pronounced for object control skills (i.e., ball skills), clinical assessment measures, and movements of the upper extremities or the whole body. The second meta-analysis investigated whether gross motor and social skills are related in ASD, synthesizing data from 21 studies representing 654 autistic individuals. Findings revealed a modest but significant overall correlation between gross motor and social skills in ASD (r = 0.27). Collectively, results support the conclusion that motor deficits are tied to the core symptoms of ASD. Further research is needed to test the causality and directionality of this relationship. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Computational Measurement of Motor Imitation and Imitative Learning Differences in Autism Spectrum Disorder: Computational Motor Imitation Measurement in ASD

Motor imitation is a critical developmental skill area that has been strongly and specifically linked to autism spectrum disorder (ASD). However, methodological variability across studies has precluded a clear understanding of the extent and impact of imitation differences in ASD, underscoring a need for more automated, granular measurement approaches that offer greater precision and consistency. In this paper, we investigate the utility of a novel motor imitation measurement approach for accurately differentiating between youth with ASD and typically developing (TD) youth. Findings indicate that youth with ASD imitate body movements significantly differently from TD youth upon repeated administration of a brief, simple task, and that a classifier based on body coordination features derived from this task can differentiate between autistic and TD youth with 82% accuracy. Our method illustrates that group differences are driven not only by interpersonal coordination with the imitated video stimulus, but also by intrapersonal coordination. Comparison of 2D and 3D tracking shows that both approaches achieve the same classification accuracy of 82%, which is highly promising with regard to scalability for larger samples and a range of non-laboratory settings. This work adds to a rapidly growing literature highlighting the promise of computational behavior analysis for detecting and characterizing motor differences in ASD and identifying potential motor biomarkers.

Motor Skill Differences in Autism Spectrum Disorder: a Clinically Focused Review

PURPOSE OF REVIEW

This review synthesizes recent, clinically relevant findings on the scope, significance, and centrality of motor skill differences in autism spectrum disorder (ASD).

RECENT FINDINGS

Motor challenges in ASD are pervasive, clinically meaningful, and highly underrecognized, with up to 87% of the autistic population affected but only a small percentage receiving motor-focused clinical care. Across development, motor differences are associated with both core autism symptoms and broader functioning, though the precise nature of those associations and the specificity of motor profiles to ASD remain unestablished. Findings suggest that motor difficulties in ASD are quantifiable and treatable, and that detection and intervention efforts targeting motor function may also positively influence social communication. Recent evidence supports a need for explicit recognition of motor impairment within the diagnostic framework of ASD as a clinical specifier. Motor differences in ASD warrant greater clinical attention and routine incorporation into screening, evaluation, and treatment planning.

Interactional Synchrony and Its Association with Social and Communication Ability in Children With and Without Autism Spectrum Disorder

Social partners tend to coordinate their behaviors in time. This "interactional synchrony" is associated with a host of positive social outcomes, making it ripe for study in autism spectrum disorder (ASD). Twenty children with ASD and 17 typically developing (TD) children participated in conversations with familiar and unfamiliar adults. Conversations were rated for movement synchrony and verbal synchrony, and mothers completed measures regarding children's everyday social and communication skills. Children with ASD exhibited less interactional synchrony, with familiar and unfamiliar partners, than TD peers. Beyond group-level differences, interactional synchrony negatively correlated with autism symptom severity, and predicted dimensional scores on established social and communication measures. Results suggest that disrupted interactional synchrony may be associated with impaired social functioning in ASD.

Discovering Synchronized Subsets of Sequences: A Large Scale Solution

Finding the largest subset of sequences (i.e., time series) that are correlated above a certain threshold, within large datasets, is of significant interest for computer vision and pattern recognition problems across domains, including behavior analysis, computational biology, neuroscience, and finance. Maximal clique algorithms can be used to solve this problem, but they are not scalable. We present an approximate, but highly efficient and scalable, method that represents the search space as a union of sets called ϵ-expanded clusters, one of which is theoretically guaranteed to contain the largest subset of synchronized sequences. The method finds synchronized sets by fitting a Euclidean ball on ϵ-expanded clusters, using Jung's theorem. We validate the method on data from the three distinct domains of facial behavior analysis, finance, and neuroscience, where we respectively discover the synchrony among pixels of face videos, stock market item prices, and dynamic brain connectivity data. Experiments show that our method produces results comparable to, but up to 300 times faster than, maximal clique algorithms, with speed gains increasing exponentially with the number of input sequences.

Can Facial Pose and Expression Be Separated with Weak Perspective Camera?

Separating facial pose and expression within images requires a camera model for 3D-to-2D mapping. The weak perspective (WP) camera has been the most popular choice; it is the default, if not the only option, in state-of-the-art facial analysis methods and software. WP camera is justified by the supposition that its errors are negligible when the subjects are relatively far from the camera, yet this claim has never been tested despite nearly 20 years of research. This paper critically examines the suitability of WP camera for separating facial pose and expression. First, we theoretically show that WP causes pose-expression ambiguity, as it leads to estimation of spurious expressions. Next, we experimentally quantify the magnitude of spurious expressions. Finally, we test whether spurious expressions have detrimental effects on a common facial analysis application, namely Action Unit (AU) detection. Contrary to conventional wisdom, we find that severe pose-expression ambiguity exists even when subjects are not close to the camera, leading to large false positive rates in AU detection. We also demonstrate that the magnitude and characteristics of spurious expressions depend on the point distribution model used to model the expressions. Our results suggest that common assumptions about WP need to be revisited in facial expression modeling, and that facial analysis software should encourage and facilitate the use of the true camera model whenever possible.

Computer Vision Analysis of Reduced Interpersonal Affect Coordination in Youth With Autism Spectrum Disorder

Atypical social-emotional reciprocity is a core feature of autism spectrum disorder (ASD) but can be difficult to operationalize. One measurable manifestation of reciprocity may be interpersonal coordination, the tendency to align the form and timing of one's behaviors (including facial affect) with others. Interpersonal affect coordination facilitates sharing and understanding of emotional cues, and there is evidence that it is reduced in ASD. However, most research has not measured this process in true social contexts, due in part to a lack of tools for measuring dynamic facial expressions over the course of an interaction. Automated facial analysis via computer vision provides an efficient, granular, objective method for measuring naturally occurring facial affect and coordination. Youth with ASD and matched typically developing youth participated in cooperative conversations with their mothers and unfamiliar adults. Time-synchronized videos were analyzed with an open-source computer vision toolkit for automated facial analysis, for the presence and intensity of facial movements associated with positive affect. Both youth and adult conversation partners exhibited less positive affect during conversations when the youth partner had ASD. Youth with ASD also engaged in less affect coordination over the course of conversations. When considered dimensionally across youth with and without ASD, affect coordination significantly predicted scores on rating scales of autism-related social atypicality, adaptive social skills, and empathy. Findings suggest that affect coordination is an important interpersonal process with implications for broader social-emotional functioning. This preliminary study introduces a promising novel method for quantifying moment-to-moment facial expression and emotional reciprocity during natural interactions. LAY SUMMARY: This study introduces a novel, automated method for measuring social-emotional reciprocity during natural conversations, which may improve assessment of this core autism diagnostic behavior. We used computerized methods to measure facial affect and the degree of affect coordination between conversation partners. Youth with autism displayed reduced affect coordination, and reduced affect coordination predicted lower scores on measures of broader social-emotional skills.

Sex differences in the first impressions made by girls and boys with autism

BACKGROUND

Individuals with autism spectrum disorder (ASD) are characterized by social communication challenges and repetitive behaviors that may be quickly detected by experts (Autism Res 10:653-62, 2017; American Psychiatric Association, Diagnostic and statistical manual of mental disorders, 2013). Recent research suggests that even naïve non-experts judge a variety of human dimensions using narrow windows of experience called "first impressions." Growing recognition of sex differences in a variety of observable behaviors in ASD, combined with research showing that some autistic girls and women may "camouflage" outward symptoms, suggests it may be more difficult for naïve conversation partners to detect ASD symptoms in girls. Here, we explore the first impressions made by boys and girls with ASD and typically developing (TD) peers.

METHODS

Ninety-three school-aged children with ASD or TD were matched on IQ; autistic girls and boys were additionally matched on autism symptom severity using the ADOS-2. Participants completed a 5-minute "get-to-know-you" conversation with a new young adult acquaintance. Immediately after the conversation, confederates rated participants on a variety of dimensions. Our primary analysis compared conversation ratings between groups (ASD boys, ASD girls, TD boys, TD girls).

RESULTS

Autistic girls were rated more positively than autistic boys by novel conversation partners (better perceived social communication ability), despite comparable autism symptom severity as rated by expert clinicians (equivalent true social communication ability). Boys with ASD were rated more negatively than typical boys and typical girls by novel conversation partners as well as expert clinicians. There was no significant difference in the first impressions made by autistic girls compared to typical girls during conversations with a novel conversation partner, but autistic girls were rated lower than typical girls by expert clinicians.

LIMITATIONS

This study cannot speak to the ways in which first impressions may differ for younger children, adults, or individuals who are not verbally fluent; in addition, there were more autistic boys than girls in our sample, making it difficult to detect small effects.

CONCLUSIONS

First impressions made during naturalistic conversations with non-expert conversation partners could-in combination with clinical ratings and parent report-shed light on the nature and effects of behavioral differences between girls and boys on the autism spectrum.

A Lifespan Approach to Patient-Reported Outcomes and Quality of Life for People on the Autism Spectrum

Autistic self-advocates, family members, and community organizations have called for greater emphasis on enhancing quality of life (QoL) for people with autism. Doing this is critical to understand how QoL unfolds across the life course and to clarify whether gender affects QoL, health, and functioning for people with autism. The purpose of this study was to curate and test a lifespan QoL measurement tool using freely available and well-constructed National Institutes of Health Parent-Reported Outcomes Measurement Information System (PROMIS). To develop the PROMIS Autism Battery-Lifespan (PAB-L), we identified PROMIS scales relevant for autism, reviewed each item, consulted with a panel of autism experts, and elicited feedback from autistic people and family members. This battery provides a comprehensive portrait of QoL for children ages 5-13 (through parent proxy), teens 14-17 (parent proxy and self-report), and adults 18-65 (self-report) with autism compared to the general population. Participants and parent informants (N = 912) recruited through a children's hospital and nationwide U.S. autism research registry completed the PAB-L online. Results indicate that compared to general population norms, people with autism of all ages (or their proxies) reported less desirable outcomes and lower QoL across all domains. Women and girls experienced greater challenges in some areas compared to men and boys with autism. The PAB-L appears to be a feasible and acceptable method for assessing patient-reported outcomes and QoL for autistic people across the life course. Autism Res 2020, 13: 970-987. © 2020 International Society for Autism Research, Wiley Periodicals, Inc. LAY SUMMARY: We developed a survey to measure the quality of life of children, teens, and adults with autism using free National Institutes of Health PROMIS questionnaires. People with autism and family members rated the PROMIS Autism Battery-Lifespan as useful and important. Some reported a good quality of life, while many reported that their lives were not going as well as they wanted. Women and girls reported more challenges in some areas of life than men and boys.

Inequality-Constrained and Robust 3D Face Model Fitting

Fitting 3D morphable models (3DMMs) on faces is a well-studied problem, motivated by various industrial and research applications. 3DMMs express a 3D facial shape as a linear sum of basis functions. The resulting shape, however, is a plausible face only when the basis coefficients take values within limited intervals. Methods based on unconstrained optimization address this issue with a weighted ℓ ₂ penalty on coefficients; however, determining the weight of this penalty is difficult, and the existence of a single weight that works universally is questionable. We propose a new formulation that does not require the tuning of any weight parameter. Specifically, we formulate 3DMM fitting as an inequality-constrained optimization problem, where the primary constraint is that basis coefficients should not exceed the interval that is learned when the 3DMM is constructed. We employ additional constraints to exploit sparse landmark detectors, by forcing the facial shape to be within the error bounds of a reliable detector. To enable operation "in-the-wild", we use a robust objective function, namely Gradient Correlation. Our approach performs comparably with deep learning (DL) methods on "in-the-wild" data that have inexact ground truth, and better than DL methods on more controlled data with exact ground truth. Since our formulation does not require any learning, it enjoys a versatility that allows it to operate with multiple frames of arbitrary sizes. This study's results encourage further research on 3DMM fitting with inequality-constrained optimization methods, which have been unexplored compared to unconstrained methods.