Computational Methods to Measure Patterns of Gaze in Toddlers With Autism Spectrum Disorder

A Review on Autism Spectrum Disorder Screening by Artificial Intelligence Methods

PURPOSE

With the increasing prevalence of autism spectrum disorders (ASD), the importance of early screening and diagnosis has been subject to considerable discussion. Given the subtle differences between ASD children and typically developing children during the early stages of development, it is imperative to investigate the utilization of automatic recognition methods powered by artificial intelligence. We aim to summarize the research work on this topic and sort out the markers that can be used for identification.

METHODS

We searched the papers published in the Web of Science, PubMed, Scopus, Medline, SpringerLink, Wiley Online Library, and EBSCO databases from 1st January 2013 to 13th November 2023, and 43 articles were included.

RESULTS

These articles mainly divided recognition markers into five categories: gaze behaviors, facial expressions, motor movements, voice features, and task performance. Based on the above markers, the accuracy of artificial intelligence screening ranged from 62.13 to 100%, the sensitivity ranged from 69.67 to 100%, the specificity ranged from 54 to 100%.

CONCLUSION

Therefore, artificial intelligence recognition holds promise as a tool for identifying children with ASD. However, it still needs to continually enhance the screening model and improve accuracy through multimodal screening, thereby facilitating timely intervention and treatment.

Smartphone-based gaze estimation for in-home autism research

Atypical gaze patterns are a promising biomarker of autism spectrum disorder. To measure gaze accurately, however, it typically requires highly controlled studies in the laboratory using specialized equipment that is often expensive, thereby limiting the scalability of these approaches. Here we test whether a recently developed smartphone-based gaze estimation method could overcome such limitations and take advantage of the ubiquity of smartphones. As a proof-of-principle, we measured gaze while a small sample of well-assessed autistic participants and controls watched videos on a smartphone, both in the laboratory (with lab personnel) and in remote home settings (alone). We demonstrate that gaze data can be efficiently collected, in-home and longitudinally by participants themselves, with sufficiently high accuracy (gaze estimation error below 1° visual angle on average) for quantitative, feature-based analysis. Using this approach, we show that autistic individuals have reduced gaze time on human faces and longer gaze time on non-social features in the background, thereby reproducing established findings in autism using just smartphones and no additional hardware. Our approach provides a foundation for scaling future research with larger and more representative participant groups at vastly reduced cost, also enabling better inclusion of underserved communities.

The role of eye movement signals in non-invasive brain-computer interface typing system

Brain-Computer Interfaces (BCIs) have shown great potential in providing communication and control for individuals with severe motor disabilities. However, traditional BCIs that rely on electroencephalography (EEG) signals suffer from low information transfer rates and high variability across users. Recently, eye movement signals have emerged as a promising alternative due to their high accuracy and robustness. Eye movement signals are the electrical or mechanical signals generated by the movements and behaviors of the eyes, serving to denote the diverse forms of eye movements, such as fixations, smooth pursuit, and other oculomotor activities like blinking. This article presents a review of recent studies on the development of BCI typing systems that incorporate eye movement signals. We first discuss the basic principles of BCI and the recent advancements in text entry. Then, we provide a comprehensive summary of the latest advancements in BCI typing systems that leverage eye movement signals. This includes an in-depth analysis of hybrid BCIs that are built upon the integration of electrooculography (EOG) and eye tracking technology, aiming to enhance the performance and functionality of the system. Moreover, we highlight the advantages and limitations of different approaches, as well as potential future directions. Overall, eye movement signals hold great potential for enhancing the usability and accessibility of BCI typing systems, and further research in this area could lead to more effective communication and control for individuals with motor disabilities.

Identification of autism spectrum disorder based on electroencephalography: A systematic review

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by difficulties in social communication and repetitive and stereotyped behaviors. According to the World Health Organization, about 1 in 100 children worldwide has autism. With the global prevalence of ASD, timely and accurate diagnosis has been essential in enhancing the intervention effectiveness for ASD children. Traditional ASD diagnostic methods rely on clinical observations and behavioral assessment, with the disadvantages of time-consuming and lack of objective biological indicators. Therefore, automated diagnostic methods based on machine learning and deep learning technologies have emerged and become significant since they can achieve more objective, efficient, and accurate ASD diagnosis. Electroencephalography (EEG) is an electrophysiological monitoring method that records changes in brain spontaneous potential activity, which is of great significance for identifying ASD children. By analyzing EEG data, it is possible to detect abnormal synchronous neuronal activity of ASD children. This paper gives a comprehensive review of the EEG-based ASD identification using traditional machine learning methods and deep learning approaches, including their merits and potential pitfalls. Additionally, it highlights the challenges and the opportunities ahead in search of more effective and efficient methods to automatically diagnose autism based on EEG signals, which aims to facilitate automated ASD identification.

Digitally Diagnosing Multiple Developmental Delays Using Crowdsourcing Fused With Machine Learning: Protocol for a Human-in-the-Loop Machine Learning Study

BACKGROUND

A considerable number of minors in the United States are diagnosed with developmental or psychiatric conditions, potentially influenced by underdiagnosis factors such as cost, distance, and clinician availability. Despite the potential of digital phenotyping tools with machine learning (ML) approaches to expedite diagnoses and enhance diagnostic services for pediatric psychiatric conditions, existing methods face limitations because they use a limited set of social features for prediction tasks and focus on a single binary prediction, resulting in uncertain accuracies.

OBJECTIVE

This study aims to propose the development of a gamified web system for data collection, followed by a fusion of novel crowdsourcing algorithms with ML behavioral feature extraction approaches to simultaneously predict diagnoses of autism spectrum disorder and attention-deficit/hyperactivity disorder in a precise and specific manner.

METHODS

The proposed pipeline will consist of (1) gamified web applications to curate videos of social interactions adaptively based on the needs of the diagnostic system, (2) behavioral feature extraction techniques consisting of automated ML methods and novel crowdsourcing algorithms, and (3) the development of ML models that classify several conditions simultaneously and that adaptively request additional information based on uncertainties about the data.

RESULTS

A preliminary version of the web interface has been implemented, and a prior feature selection method has highlighted a core set of behavioral features that can be targeted through the proposed gamified approach.

CONCLUSIONS

The prospect for high reward stems from the possibility of creating the first artificial intelligence-powered tool that can identify complex social behaviors well enough to distinguish conditions with nuanced differentiators such as autism spectrum disorder and attention-deficit/hyperactivity disorder.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/52205.

Autism Caregiver Coaching in Africa (ACACIA): Protocol for a type 1-hybrid effectiveness-implementation trial

BACKGROUND

While early autism intervention can significantly improve outcomes, gaps in implementation exist globally. These gaps are clearest in Africa, where forty percent of the world's children will live by 2050. Task-sharing early intervention to non-specialists is a key implementation strategy, given the lack of specialists in Africa. Naturalistic Developmental Behavioral Interventions (NDBI) are a class of early autism intervention that can be delivered by caregivers. As a foundational step to address the early autism intervention gap, we adapted a non-specialist delivered caregiver coaching NDBI for the South African context, and pre-piloted this cascaded task-sharing approach in an existing system of care.

OBJECTIVES

First, we will test the effectiveness of the caregiver coaching NDBI compared to usual care. Second, we will describe coaching implementation factors within the Western Cape Department of Education in South Africa.

METHODS

This is a type 1 effectiveness-implementation hybrid design; assessor-blinded, group randomized controlled trial. Participants include 150 autistic children (18-72 months) and their caregivers who live in Cape Town, South Africa, and those involved in intervention implementation. Early Childhood Development practitioners, employed by the Department of Education, will deliver 12, one hour, coaching sessions to the intervention group. The control group will receive usual care. Distal co-primary outcomes include the Communication Domain Standard Score (Vineland Adaptive Behavior Scales, Third Edition) and the Language and Communication Developmental Quotient (Griffiths Scales of Child Development, Third Edition). Proximal secondary outcome include caregiver strategies measured by the sum of five items from the Joint Engagement Rating Inventory. We will describe key implementation determinants.

RESULTS

Participant enrolment started in April 2023. Estimated primary completion date is March 2027.

CONCLUSION

The ACACIA trial will determine whether a cascaded task-sharing intervention delivered in an educational setting leads to meaningful improvements in communication abilities of autistic children, and identify implementation barriers and facilitators.

TRIAL REGISTRATION

NCT05551728 in Clinical Trial Registry (https://clinicaltrials.gov).

Assessment of Autism Spectrum Disorder

Autism Spectrum Disorder (ASD) is a neurodevelopmental condition characterized by challenges in social interaction and communication and the presence of restricted interests and repetitive behaviors. The importance of early detection of ASD and subsequent early intervention is well documented. Efforts have been made over the years to clarify ASD diagnostic criteria and develop predictive, accurate screening tools and evidence-based, standardized diagnostic instruments to aid in the identification of ASD. In this article, we review the most recent changes in ASD diagnostic criteria in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, Text Revision, summarize evidence-based instruments for ASD screening and diagnostic evaluations as well as the assessment of co-occurring conditions in ASD, the impact of COVID-19 on ASD assessment, and directions for future research in the field of ASD assessment.

Remote, tablet-based assessment of gaze following: a nationwide infant twin study

Introduction

Much of our understanding of infant psychological development relies on an in-person, laboratory-based assessment. This limits research generalizability, scalability, and equity in access. One solution is the development of new, remotely deployed assessment tools that do not require real-time experimenter supervision.

Methods

The current nationwide (Sweden) infant twin study assessed participants remotely via their caregiver's tablets (N = 104, ages 3 to 17 months). To anchor our findings in previous research, we used a gaze-following task where experimental and age effects are well established.

Results

Closely mimicking results from conventional eye tracking, we found that a full head movement elicited more gaze following than isolated eye movements. Furthermore, predictably, we found that older infants followed gaze more frequently than younger infants. Finally, while we found no indication of genetic contributions to gaze-following accuracy, the latency to disengage from the gaze cue and orient toward a target was significantly more similar in monozygotic twins than in dizygotic twins, an indicative of heritability.

Discussion

Together, these results highlight the potential of remote assessment of infants' psychological development, which can improve generalizability, inclusion, and scalability in developmental research.

Early detection of autism using digital behavioral phenotyping

Early detection of autism, a neurodevelopmental condition associated with challenges in social communication, ensures timely access to intervention. Autism screening questionnaires have been shown to have lower accuracy when used in real-world settings, such as primary care, as compared to research studies, particularly for children of color and girls. Here we report findings from a multiclinic, prospective study assessing the accuracy of an autism screening digital application (app) administered during a pediatric well-child visit to 475 (17-36 months old) children (269 boys and 206 girls), of which 49 were diagnosed with autism and 98 were diagnosed with developmental delay without autism. The app displayed stimuli that elicited behavioral signs of autism, quantified using computer vision and machine learning. An algorithm combining multiple digital phenotypes showed high diagnostic accuracy with the area under the receiver operating characteristic curve = 0.90, sensitivity = 87.8%, specificity = 80.8%, negative predictive value = 97.8% and positive predictive value = 40.6%. The algorithm had similar sensitivity performance across subgroups as defined by sex, race and ethnicity. These results demonstrate the potential for digital phenotyping to provide an objective, scalable approach to autism screening in real-world settings. Moreover, combining results from digital phenotyping and caregiver questionnaires may increase autism screening accuracy and help reduce disparities in access to diagnosis and intervention.

Autism Caregiver Coaching in Africa (ACACIA): Protocol for a type 1-hybrid effectiveness-implementation trial

Background

While early autism intervention can significantly improve outcomes, gaps in implementation exist globally. These gaps are clearest in Africa, where forty percent of the world's children will live by 2050. Task-sharing early intervention to non-specialists is a key implementation strategy, given the lack of specialists in Africa. Naturalistic Developmental Behavioral Interventions (NDBI) are a class of early autism intervention that can be delivered by caregivers. As a foundational step to address the early autism intervention gap, we adapted a non-specialist delivered caregiver coaching NDBI for the South African context, and pre-piloted this cascaded task-sharing approach in an existing system of care.

Objectives

First, we will test the effectiveness of the caregiver coaching NDBI compared to usual care. Second, we will describe coaching implementation factors within the Western Cape Department of Education in South Africa.

Methods

This is a type 1 effectiveness-implementation hybrid design; assessor-blinded, group randomized controlled trial. Participants include 150 autistic children (18-72 months) and their caregivers who live in Cape Town, South Africa, and those involved in intervention implementation. Early Childhood Development practitioners, employed by the Department of Education, will deliver 12, one hour, coaching sessions to the intervention group. The control group will receive usual care. Distal co-primary outcomes include the Communication Domain Standard Score (Vineland Adaptive Behavior Scales, Third Edition) and the Language and Communication Developmental Quotient (Griffiths Scales of Child Development, Third Edition). Proximal secondary outcome include caregiver strategies measured by the sum of five items from the Joint Engagement Rating Inventory. We will describe key implementation determinants.

Results

Participant enrolment started in April 2023. Estimated primary completion date is March 2027.

Conclusion

The ACACIA trial will determine whether a cascaded task-sharing intervention delivered in an educational setting leads to meaningful improvements in communication abilities of autistic children, and identify implementation barriers and facilitators.

A Review of and Roadmap for Data Science and Machine Learning for the Neuropsychiatric Phenotype of Autism

Autism spectrum disorder (autism) is a neurodevelopmental delay that affects at least 1 in 44 children. Like many neurological disorder phenotypes, the diagnostic features are observable, can be tracked over time, and can be managed or even eliminated through proper therapy and treatments. However, there are major bottlenecks in the diagnostic, therapeutic, and longitudinal tracking pipelines for autism and related neurodevelopmental delays, creating an opportunity for novel data science solutions to augment and transform existing workflows and provide increased access to services for affected families. Several efforts previously conducted by a multitude of research labs have spawned great progress toward improved digital diagnostics and digital therapies for children with autism. We review the literature on digital health methods for autism behavior quantification and beneficial therapies using data science. We describe both case-control studies and classification systems for digital phenotyping. We then discuss digital diagnostics and therapeutics that integrate machine learning models of autism-related behaviors, including the factors that must be addressed for translational use. Finally, we describe ongoing challenges and potential opportunities for the field of autism data science. Given the heterogeneous nature of autism and the complexities of the relevant behaviors, this review contains insights that are relevant to neurological behavior analysis and digital psychiatry more broadly.

Enhancing early autism prediction based on electronic records using clinical narratives

Recent work has shown that predictive models can be applied to structured electronic health record (EHR) data to stratify autism likelihood from an early age (<1 year). Integrating clinical narratives (or notes) with structured data has been shown to improve prediction performance in other clinical applications, but the added predictive value of this information in early autism prediction has not yet been explored. In this study, we aimed to enhance the performance of early autism prediction by using both structured EHR data and clinical narratives. We built models based on structured data and clinical narratives separately, and then an ensemble model that integrated both sources of data. We assessed the predictive value of these models from Duke University Health System over a 14-year span to evaluate ensemble models predicting later autism diagnosis (by age 4 years) from data collected from ages 30 to 360 days. Our sample included 11,750 children above by age 3 years (385 meeting autism diagnostic criteria). The ensemble model for autism prediction showed superior performance and at age 30 days achieved 46.8% sensitivity (95% confidence interval, CI: 22.0%, 52.9%), 28.0% positive predictive value (PPV) at high (90%) specificity (CI: 2.0%, 33.1%), and AUC4 (with at least 4-year follow-up for controls) reaching 0.769 (CI: 0.715, 0.811). Prediction by 360 days achieved 44.5% sensitivity (CI: 23.6%, 62.9%), and 13.7% PPV at high (90%) specificity (CI: 9.6%, 18.9%), and AUC4 reaching 0.797 (CI: 0.746, 0.840). Results show that incorporating clinical narratives in early autism prediction achieved promising accuracy by age 30 days, outperforming models based on structured data only. Furthermore, findings suggest that additional features learned from clinician narratives might be hypothesis generating for understanding early development in autism.

Using machine-learning methods to identify early-life predictors of 11-year language outcome

BACKGROUND

Language is foundational for neurodevelopment and quality of life, but an estimated 10% of children have a language disorder at age 5. Many children shift between classifications of typical and low language if assessed at multiple times in the early years, making it difficult to identify which children will have persisting difficulties and benefit most from support. This study aims to identify a parsimonious set of preschool indicators that predict language outcomes in late childhood, using data from the population-based Early Language in Victoria Study (n = 839).

METHODS

Parents completed surveys about their children at ages 8, 12, 24, and 36 months. At 11 years, children were assessed using the Clinical Evaluation of Language Fundamentals 4th Edition (CELF-4). We used random forests to identify which of the 1990 parent-reported questions best predict children's 11-year language outcome (CELF-4 score ≤81 representing low language) and used SuperLearner to estimate the accuracy of the constrained sets of questions.

RESULTS

At 24 months, seven predictors relating to vocabulary, symbolic play, pragmatics and behavior yielded 73% sensitivity (95% CI: 57, 85) and 77% specificity (95% CI: 74, 80) for predicting low language at 11 years. [Corrections made on 5 May 2023, after first online publication: In the preceding sentence 'motor skills' has been corrected to 'behavior' in this version.] At 36 months, 7 predictors relating to morphosyntax, vocabulary, parent-child interactions, and parental stress yielded 75% sensitivity (95% CI: 58, 88) and 85% specificity (95% CI: 81, 87). Measures at 8 and 12 months yielded unsatisfactory accuracy.

CONCLUSIONS

We identified two short sets of questions that predict language outcomes at age 11 with fair accuracy. Future research should seek to replicate results in a separate cohort.

Relationship between quantitative digital behavioral features and clinical profiles in young autistic children

Early behavioral markers for autism include differences in social attention and orienting in response to one's name when called, and differences in body movements and motor abilities. More efficient, scalable, objective, and reliable measures of these behaviors could improve early screening for autism. This study evaluated whether objective and quantitative measures of autism-related behaviors elicited from an app (SenseToKnow) administered on a smartphone or tablet and measured via computer vision analysis (CVA) are correlated with standardized caregiver-report and clinician administered measures of autism-related behaviors and cognitive, language, and motor abilities. This is an essential step in establishing the concurrent validity of a digital phenotyping approach. In a sample of 485 toddlers, 43 of whom were diagnosed with autism, we found that CVA-based gaze variables related to social attention were associated with the level of autism-related behaviors. Two language-related behaviors measured via the app, attention to people during a conversation and responding to one's name being called, were associated with children's language skills. Finally, performance during a bubble popping game was associated with fine motor skills. These findings provide initial support for the concurrent validity of the SenseToKnow app and its potential utility in identifying clinical profiles associated with autism. Future research is needed to determine whether the app can be used as an autism screening tool, can reliably stratify autism-related behaviors, and measure changes in autism-related behaviors over time.

Blink rate and facial orientation reveal distinctive patterns of attentional engagement in autistic toddlers: a digital phenotyping approach

Differences in social attention are well-documented in autistic individuals, representing one of the earliest signs of autism. Spontaneous blink rate has been used to index attentional engagement, with lower blink rates reflecting increased engagement. We evaluated novel methods using computer vision analysis (CVA) for automatically quantifying patterns of attentional engagement in young autistic children, based on facial orientation and blink rate, which were captured via mobile devices. Participants were 474 children (17-36 months old), 43 of whom were diagnosed with autism. Movies containing social or nonsocial content were presented via an iPad app, and simultaneously, the device's camera recorded the children's behavior while they watched the movies. CVA was used to extract the duration of time the child oriented towards the screen and their blink rate as indices of attentional engagement. Overall, autistic children spent less time facing the screen and had a higher mean blink rate compared to neurotypical children. Neurotypical children faced the screen more often and blinked at a lower rate during the social movies compared to the nonsocial movies. In contrast, autistic children faced the screen less often during social movies than during nonsocial movies and showed no differential blink rate to social versus nonsocial movies.

Sensitivity and Specificity of the Modified Checklist for Autism in Toddlers (Original and Revised): A Systematic Review and Meta-analysis

Importance

The Modified Checklist for Autism in Toddlers (M-CHAT) and the M-CHAT, Revised With Follow-up (M-CHAT-R/F)-henceforth referred to as M-CHAT(-R/F)-are the most commonly used toddler screeners for autism spectrum disorder (ASD). Their use often differs from that in the original validation studies, resulting in a range of estimates of sensitivity and specificity. Also, given the variability in reports of the clinical utility of the M-CHAT(-R/F), researchers and practitioners lack guidance to inform autism screening protocols.

Objective

To synthesize variability in sensitivity and specificity of M-CHAT(-R/F) across multiple factors, including procedures for identifying missed cases, likelihood level, screening age, and single compared with repeated screenings.

Data Sources

A literature search was conducted with PubMed, Web of Science, and Scopus to identify studies published between January 1, 2001, and August 31, 2022.

Study Selection

Articles were included if the studies used the M-CHAT(-R/F) (ie, original or revised version) to identify new ASD cases, were published in English-language peer-reviewed journals, included at least 10 ASD cases, reported procedures for false-negative case identification, screened children by 48 months, and included information (or had information provided by authors when contacted) needed to conduct the meta-analysis.

Data Extraction and Synthesis

The systematic review and meta-analysis was conducted within the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline. The Quality Assessment of Diagnostic Accuracy Studies-2 tool evaluated bias in sample selection. Data extraction and quality assessment were performed by 2 authors independently. The overall diagnostic accuracy of the M-CHAT(-R/F) was assessed with the hierarchic summary receiver operating characteristic (HSROC) model.

Main Outcomes and Measures

Sensitivity, specificity, diagnostic odds ratios, and HSROC curves of M-CHAT(-R/F).

Results

The review included 50 studies with 51 samples. The pooled sensitivity of M-CHAT(-R/F) was 0.83 (95% CI, 0.77-0.88), and the pooled specificity was 0.94 (95% CI, 0.89-0.97). Heterogeneity analyses revealed greater diagnostic accuracy for low- vs high-likelihood samples, a concurrent vs prospective case confirmation strategy, a large vs small sample size, use of M-CHAT(-R/F) Follow-up, and non-English vs English only.

Conclusions and Relevance

Overall, results of this study suggest the utility of the M-CHAT(-R/F) as an ASD screener. The wide variability in psychometric properties of M-CHAT(-R/F) highlights differences in screener use that should be considered in research and practice.

Exploring Complexity of Facial Dynamics in Autism Spectrum Disorder

Atypical facial expression is one of the early symptoms of autism spectrum disorder (ASD) characterized by reduced regularity and lack of coordination of facial movements. Automatic quantification of these behaviors can offer novel biomarkers for screening, diagnosis, and treatment monitoring of ASD. In this work, 40 toddlers with ASD and 396 typically developing toddlers were shown developmentally-appropriate and engaging movies presented on a smart tablet during a well-child pediatric visit. The movies consisted of social and non-social dynamic scenes designed to evoke certain behavioral and affective responses. The front-facing camera of the tablet was used to capture the toddlers' face. Facial landmarks' dynamics were then automatically computed using computer vision algorithms. Subsequently, the complexity of the landmarks' dynamics was estimated for the eyebrows and mouth regions using multiscale entropy. Compared to typically developing toddlers, toddlers with ASD showed higher complexity (i.e., less predictability) in these landmarks' dynamics. This complexity in facial dynamics contained novel information not captured by traditional facial affect analyses. These results suggest that computer vision analysis of facial landmark movements is a promising approach for detecting and quantifying early behavioral symptoms associated with ASD.

Eye-tracking measures of social versus nonsocial attention are related to level of social engagement during naturalistic caregiver-child interactions in autistic children

Eye-tracking (ET) measures indexing social attention have been proposed as sensitive measures related to autism, but less is known about the relationship between social and nonsocial attention and naturalistic measures of social engagement and whether sex moderates this relationship. This study investigated ET measures of social attention as predictors of social engagement during a naturalistic caregiver-child interaction (CCI). Participants included 132, 2-7-year-old autistic children (77% male) and their caregivers. Participants engaged in a CCI and an ET task in which they viewed a video of an actor making dyadic bids toward the child with toys in the background. Pearson correlations and multiple regression analyzes revealed that ET measures correlated with social engagement behaviors, including degree of attention to the caregiver and objects, joint engagement with the caregiver, and language-based joint engagement. Children who spent more time looking at toys were more likely to be unengaged during social interaction. Those who spent more time looking at the actor's mouth were more likely to engage in coordinated play with and without language. Sex moderated the relationship between time looking at toys and unengagement during play; males who spent more time looking at toys spent more time unengaged during play, whereas females who spent more time looking at toys spent less time unengaged during play. Overall, ET measures of social and nonsocial attention correlated with the level of social engagement during naturalistic play, with some sex differences. Eye-tracking measures that predict interaction patterns may provide insight into promoting social engagement between caregivers and their autistic children and can inform outcome monitoring and intervention development.

Digitally Diagnosing Multiple Developmental Delays using Crowdsourcing fused with Machine Learning: A Research Protocol

Background

Roughly 17% percent of minors in the United States aged 3 through 17 years have a diagnosis of one or more developmental or psychiatric conditions, with the true prevalence likely being higher due to underdiagnosis in rural areas and for minority populations. Unfortunately, timely diagnostic services are inaccessible to a large portion of the United States and global population due to cost, distance, and clinician availability. Digital phenotyping tools have the potential to shorten the time-to-diagnosis and to bring diagnostic services to more people by enabling accessible evaluations. While automated machine learning (ML) approaches for detection of pediatric psychiatry conditions have garnered increased research attention in recent years, existing approaches use a limited set of social features for the prediction task and focus on a single binary prediction.

Objective

I propose the development of a gamified web system for data collection followed by a fusion of novel crowdsourcing algorithms with machine learning behavioral feature extraction approaches to simultaneously predict diagnoses of Autism Spectrum Disorder (ASD) and Attention-Deficit/Hyperactivity Disorder (ADHD) in a precise and specific manner.

Methods

The proposed pipeline will consist of: (1) a gamified web applications to curate videos of social interactions adaptively based on needs of the diagnostic system, (2) behavioral feature extraction techniques consisting of automated ML methods and novel crowdsourcing algorithms, and (3) development of ML models which classify several conditions simultaneously and which adaptively request additional information based on uncertainties about the data.

Conclusions

The prospective for high reward stems from the possibility of creating the first AI-powered tool which can identify complex social behaviors well enough to distinguish conditions with nuanced differentiators such as ASD and ADHD.

Prediction of autism in infants: progress and challenges

Autism spectrum disorder (henceforth autism) is a neurodevelopmental condition that can be reliably diagnosed in children by age 18-24 months. Prospective longitudinal studies of infants aged 1 year and younger who are later diagnosed with autism are elucidating the early developmental course of autism and identifying ways of predicting autism before diagnosis is possible. Studies that use MRI, EEG, and near-infrared spectroscopy have identified differences in brain development in infants later diagnosed with autism compared with infants without autism. Retrospective studies of infants younger than 1 year who received a later diagnosis of autism have also showed an increased prevalence of health conditions, such as sleep disorders, gastrointestinal disorders, and vision problems. Behavioural features of infants later diagnosed with autism include differences in attention, vocalisations, gestures, affect, temperament, social engagement, sensory processing, and motor abilities. Although research findings offer insight on promising screening approaches for predicting autism in infants, individual-level predictions remain a future goal. Multiple scientific challenges and ethical questions remain to be addressed to translate research on early brain-based and behavioural predictors of autism into feasible and reliable screening tools for clinical practice.

Translating advances in developmental social neuroscience into greater access to early diagnosis in autism spectrum disorder

Early identification and diagnosis of autism spectrum disorder (ASD) is necessary to promote access to early treatment, a critical factor in optimizing children's lifetime outcomes. And yet, diagnosis is often late, delaying interventions to a time in which symptoms have aggravated and communication skills already show impairing differences. This review illustrates progress in developmental social neuroscience that shows promise in generating novel tools for objective and cost-effective early diagnosis of ASD. We focus on research of social visual engagement, which is the way infants and toddlers look at and learn from their social environment. Moment-by-moment quantification of social visual engagement is yielding measures that are beginning to approximate best-practice procedures used by experienced clinicians in the assessment of young children. This progress and potential solutions have public health import ance because experienced clinicians are limited in number, and specialized clinical assessment services tend to be lengthy, costly, and plagued by extended wait time, all of which contributing to limited access, particularly in the case of low-resource families. The research reviewed here illustrates a wider effort to advance biomarker-based measurements intended to develop better and more efficient tools and procedures for screening, diagnosing and monitoring treatment response in children with ASD. The advent of such tools could increase access to early diagnostic services and promote efficiencies in early treatment delivery, with the ultimate goal of ensuring that children with ASD are afforded the services they need to thrive.

A tablet-based game for the assessment of visual motor skills in autistic children

Increasing evidence suggests that early motor impairments are a common feature of autism. Thus, scalable, quantitative methods for measuring motor behavior in young autistic children are needed. This work presents an engaging and scalable assessment of visual-motor abilities based on a bubble-popping game administered on a tablet. Participants are 233 children ranging from 1.5 to 10 years of age (147 neurotypical children and 86 children diagnosed with autism spectrum disorder [autistic], of which 32 are also diagnosed with co-occurring attention-deficit/hyperactivity disorder [autistic+ADHD]). Computer vision analyses are used to extract several game-based touch features, which are compared across autistic, autistic+ADHD, and neurotypical participants. Results show that younger (1.5-3 years) autistic children pop the bubbles at a lower rate, and their ability to touch the bubble's center is less accurate compared to neurotypical children. When they pop a bubble, their finger lingers for a longer period, and they show more variability in their performance. In older children (3-10-years), consistent with previous research, the presence of co-occurring ADHD is associated with greater motor impairment, reflected in lower accuracy and more variable performance. Several motor features are correlated with standardized assessments of fine motor and cognitive abilities, as evaluated by an independent clinical assessment. These results highlight the potential of touch-based games as an efficient and scalable approach for assessing children's visual-motor skills, which can be part of a broader screening tool for identifying early signs associated with autism.

Complexity analysis of head movements in autistic toddlers

BACKGROUND

Early differences in sensorimotor functioning have been documented in young autistic children and infants who are later diagnosed with autism. Previous research has demonstrated that autistic toddlers exhibit more frequent head movement when viewing dynamic audiovisual stimuli, compared to neurotypical toddlers. To further explore this behavioral characteristic, in this study, computer vision (CV) analysis was used to measure several aspects of head movement dynamics of autistic and neurotypical toddlers while they watched a set of brief movies with social and nonsocial content presented on a tablet.

METHODS

Data were collected from 457 toddlers, 17-36 months old, during their well-child visit to four pediatric primary care clinics. Forty-one toddlers were subsequently diagnosed with autism. An application (app) displayed several brief movies on a tablet, and the toddlers watched these movies while sitting on their caregiver's lap. The front-facing camera in the tablet recorded the toddlers' behavioral responses. CV was used to measure the participants' head movement rate, movement acceleration, and complexity using multiscale entropy.

RESULTS

Autistic toddlers exhibited significantly higher rate, acceleration, and complexity in their head movements while watching the movies compared to neurotypical toddlers, regardless of the type of movie content (social vs. nonsocial). The combined features of head movement acceleration and complexity reliably distinguished the autistic and neurotypical toddlers.

CONCLUSIONS

Autistic toddlers exhibit differences in their head movement dynamics when viewing audiovisual stimuli. Higher complexity of their head movements suggests that their movements were less predictable and less stable compared to neurotypical toddlers. CV offers a scalable means of detecting subtle differences in head movement dynamics, which may be helpful in identifying early behaviors associated with autism and providing insight into the nature of sensorimotor differences associated with autism.

Analysis and novel methods for capture of normative eye-tracking data in 2.5-month old infants

Development of attention systems is essential for both cognitive and social behavior maturation. Visual behavior has been used to assess development of these attention systems. Yet, given its importance, there is a notable lack of literature detailing successful methods and procedures for using eye-tracking in early infancy to assess oculomotor and attention dynamics. Here we show that eye-tracking technology can be used to automatically record and assess visual behavior in infants as young as 2.5 months, and present normative data describing fixation and saccade behavior at this age. Features of oculomotor dynamics were analyzed from 2.5-month old infants who viewed videos depicting live action, cartoons, geometric shapes, social and non-social scenes. Of the 54 infants enrolled, 50 infants successfully completed the eye-tracking task and high-quality data was collected for 32 of those infants. We demonstrate that modifications specifically tailored for the infant population allowed for consistent tracking of pupil and corneal reflection and minimal data loss. Additionally, we found consistent fixation and saccade behaviors across the entire six-minute duration of the videos, indicating that this is a feasible task for 2.5-month old infants. Moreover, normative oculomotor metrics for a free-viewing task in 2.5-month old infants are documented for the first time as a result of this high-quality data collection.

Measuring social orienting in preschoolers with autism spectrum disorder using cartoons stimuli

Altered social orienting (SO) was proposed as the primary source of socio-communicative difficulties in autism spectrum disorder (ASD). Eye-tracking studies generally confirm a decreased SO in ASD population. However, SO has been scarcely investigated using minimally social stimuli such as cartoons. The extent to which SO might be decreased when watching cartoons is therefore unknown. Yet, it could allow for malleable and child-friendly paradigms that could be sensitive to early atypical visual preference. In this study, 90 preschoolers with ASD (age = 3.19 ± 0 .88) and 20 TD (age = 2.95 ± 1.26) watched two eye-tracking preference tasks. One Realistic task, displaying children dancing versus geometric shapes moving repetitively and a Cartoon task, displaying social and non-social cartoon stimuli with similar movements. We measured SO percentage along with refined visual exploration parameters and compared those of ASD children to TDs. In addition, we investigated their relations with behavioral measures such as symptom severity, developmental and adaptive levels. We evidenced a decreased SO percentage in ASD compared to TD children when watching the Realistic task but not the Cartoon task. We did not identify any other between groups differences. However, we identified several correlations between eye-tracking measures and developmental as well as adaptive measures within the Cartoon task. Together, our results support a preferential orientation of children with autism towards repetitively moving shapes but no decreased SO when measured with minimally social stimuli. Nonetheless, when investigating finer visual exploration parameters, even socially simple stimuli elicited atypical gaze patterns related to early developmental delay.

The Classification of Abnormal Hand Movement to Aid in Autism Detection: Machine Learning Study

Background

A formal autism diagnosis can be an inefficient and lengthy process. Families may wait several months or longer before receiving a diagnosis for their child despite evidence that earlier intervention leads to better treatment outcomes. Digital technologies that detect the presence of behaviors related to autism can scale access to pediatric diagnoses. A strong indicator of the presence of autism is self-stimulatory behaviors such as hand flapping.

Objective

This study aims to demonstrate the feasibility of deep learning technologies for the detection of hand flapping from unstructured home videos as a first step toward validation of whether statistical models coupled with digital technologies can be leveraged to aid in the automatic behavioral analysis of autism. To support the widespread sharing of such home videos, we explored privacy-preserving modifications to the input space via conversion of each video to hand landmark coordinates and measured the performance of corresponding time series classifiers.

Methods

We used the Self-Stimulatory Behavior Dataset (SSBD) that contains 75 videos of hand flapping, head banging, and spinning exhibited by children. From this data set, we extracted 100 hand flapping videos and 100 control videos, each between 2 to 5 seconds in duration. We evaluated five separate feature representations: four privacy-preserved subsets of hand landmarks detected by MediaPipe and one feature representation obtained from the output of the penultimate layer of a MobileNetV2 model fine-tuned on the SSBD. We fed these feature vectors into a long short-term memory network that predicted the presence of hand flapping in each video clip.

Results

The highest-performing model used MobileNetV2 to extract features and achieved a test F1 score of 84 (SD 3.7; precision 89.6, SD 4.3 and recall 80.4, SD 6) using 5-fold cross-validation for 100 random seeds on the SSBD data (500 total distinct folds). Of the models we trained on privacy-preserved data, the model trained with all hand landmarks reached an F1 score of 66.6 (SD 3.35). Another such model trained with a select 6 landmarks reached an F1 score of 68.3 (SD 3.6). A privacy-preserved model trained using a single landmark at the base of the hands and a model trained with the average of the locations of all the hand landmarks reached an F1 score of 64.9 (SD 6.5) and 64.2 (SD 6.8), respectively.

Conclusions

We created five lightweight neural networks that can detect hand flapping from unstructured videos. Training a long short-term memory network with convolutional feature vectors outperformed training with feature vectors of hand coordinates and used almost 900,000 fewer model parameters. This study provides the first step toward developing precise deep learning methods for activity detection of autism-related behaviors.

Large scale validation of an early-age eye-tracking biomarker of an autism spectrum disorder subtype

Few clinically validated biomarkers of ASD exist which can rapidly, accurately, and objectively identify autism during the first years of life and be used to support optimized treatment outcomes and advances in precision medicine. As such, the goal of the present study was to leverage both simple and computationally-advanced approaches to validate an eye-tracking measure of social attention preference, the GeoPref Test, among 1,863 ASD, delayed, or typical toddlers (12-48 months) referred from the community or general population via a primary care universal screening program. Toddlers participated in diagnostic and psychometric evaluations and the GeoPref Test: a 1-min movie containing side-by-side dynamic social and geometric images. Following testing, diagnosis was denoted as ASD, ASD features, LD, GDD, Other, typical sibling of ASD proband, or typical. Relative to other diagnostic groups, ASD toddlers exhibited the highest levels of visual attention towards geometric images and those with especially high fixation levels exhibited poor clinical profiles. Using the 69% fixation threshold, the GeoPref Test had 98% specificity, 17% sensitivity, 81% PPV, and 65% NPV. Sensitivity increased to 33% when saccades were included, with comparable validity across sex, ethnicity, or race. The GeoPref Test was also highly reliable up to 24 months following the initial test. Finally, fixation levels among twins concordant for ASD were significantly correlated, indicating that GeoPref Test performance may be genetically driven. As the GeoPref Test yields few false positives (~ 2%) and is equally valid across demographic categories, the current findings highlight the ability of the GeoPref Test to rapidly and accurately detect autism before the 2nd birthday in a subset of children and serve as a biomarker for a unique ASD subtype in clinical trials.

Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods

BACKGROUND

Autism spectrum disorder (ASD) is a widespread neurodevelopmental condition with a range of potential causes and symptoms. Standard diagnostic mechanisms for ASD, which involve lengthy parent questionnaires and clinical observation, often result in long waiting times for results. Recent advances in computer vision and mobile technology hold potential for speeding up the diagnostic process by enabling computational analysis of behavioral and social impairments from home videos. Such techniques can improve objectivity and contribute quantitatively to the diagnostic process.

OBJECTIVE

In this work, we evaluate whether home videos collected from a game-based mobile app can be used to provide diagnostic insights into ASD. To the best of our knowledge, this is the first study attempting to identify potential social indicators of ASD from mobile phone videos without the use of eye-tracking hardware, manual annotations, and structured scenarios or clinical environments.

METHODS

Here, we used a mobile health app to collect over 11 hours of video footage depicting 95 children engaged in gameplay in a natural home environment. We used automated data set annotations to analyze two social indicators that have previously been shown to differ between children with ASD and their neurotypical (NT) peers: (1) gaze fixation patterns, which represent regions of an individual's visual focus and (2) visual scanning methods, which refer to the ways in which individuals scan their surrounding environment. We compared the gaze fixation and visual scanning methods used by children during a 90-second gameplay video to identify statistically significant differences between the 2 cohorts; we then trained a long short-term memory (LSTM) neural network to determine if gaze indicators could be predictive of ASD.

RESULTS

Our results show that gaze fixation patterns differ between the 2 cohorts; specifically, we could identify 1 statistically significant region of fixation (P<.001). In addition, we also demonstrate that there are unique visual scanning patterns that exist for individuals with ASD when compared to NT children (P<.001). A deep learning model trained on coarse gaze fixation annotations demonstrates mild predictive power in identifying ASD.

CONCLUSIONS

Ultimately, our study demonstrates that heterogeneous video data sets collected from mobile devices hold potential for quantifying visual patterns and providing insights into ASD. We show the importance of automated labeling techniques in generating large-scale data sets while simultaneously preserving the privacy of participants, and we demonstrate that specific social engagement indicators associated with ASD can be identified and characterized using such data.

Pragmatic AI-augmentation in mental healthcare: Key technologies, potential benefits, and real-world challenges and solutions for frontline clinicians

The integration of artificial intelligence (AI) technologies into mental health holds the promise of increasing patient access, engagement, and quality of care, and of improving clinician quality of work life. However, to date, studies of AI technologies in mental health have focused primarily on challenges that policymakers, clinical leaders, and data and computer scientists face, rather than on challenges that frontline mental health clinicians are likely to face as they attempt to integrate AI-based technologies into their everyday clinical practice. In this Perspective, we describe a framework for "pragmatic AI-augmentation" that addresses these issues by describing three categories of emerging AI-based mental health technologies which frontline clinicians can leverage in their clinical practice-automation, engagement, and clinical decision support technologies. We elaborate the potential benefits offered by these technologies, the likely day-to-day challenges they may raise for mental health clinicians, and some solutions that clinical leaders and technology developers can use to address these challenges, based on emerging experience with the integration of AI technologies into clinician daily practice in other healthcare disciplines.

EYE-C: Eye-Contact Robust Detection and Analysis during Unconstrained Child-Therapist Interactions in the Clinical Setting of Autism Spectrum Disorders

The high level of heterogeneity in Autism Spectrum Disorder (ASD) and the lack of systematic measurements complicate predicting outcomes of early intervention and the identification of better-tailored treatment programs. Computational phenotyping may assist therapists in monitoring child behavior through quantitative measures and personalizing the intervention based on individual characteristics; still, real-world behavioral analysis is an ongoing challenge. For this purpose, we designed EYE-C, a system based on OpenPose and Gaze360 for fine-grained analysis of eye-contact episodes in unconstrained therapist-child interactions via a single video camera. The model was validated on video data varying in resolution and setting, achieving promising performance. We further tested EYE-C on a clinical sample of 62 preschoolers with ASD for spectrum stratification based on eye-contact features and age. By unsupervised clustering, three distinct sub-groups were identified, differentiated by eye-contact dynamics and a specific clinical phenotype. Overall, this study highlights the potential of Artificial Intelligence in categorizing atypical behavior and providing translational solutions that might assist clinical practice.

Application of Eye Tracking Technology in Medicine: A Bibliometric Analysis

Eye tracking provides a quantitative measure of eye movements during different activities. We report the results from a bibliometric analysis to investigate trends in eye tracking research applied to the study of different medical conditions. We conducted a search on the Web of Science Core Collection (WoS) database and analyzed the dataset of 2456 retrieved articles using VOSviewer and the Bibliometrix R package. The most represented area was psychiatry (503, 20.5%) followed by neuroscience (465, 18.9%) and psychology developmental (337, 13.7%). The annual scientific production growth was 11.14% and showed exponential growth with three main peaks in 2011, 2015 and 2017. Extensive collaboration networks were identified between the three countries with the highest scientific production, the USA (35.3%), the UK (9.5%) and Germany (7.3%). Based on term co-occurrence maps and analyses of sources of articles, we identified autism spectrum disorders as the most investigated condition and conducted specific analyses on 638 articles related to this topic which showed an annual scientific production growth of 16.52%. The majority of studies focused on autism used eye tracking to investigate gaze patterns with regards to stimuli related to social interaction. Our analysis highlights the widespread and increasing use of eye tracking in the study of different neurological and psychiatric conditions.

Deep Learning Approach for Screening Autism Spectrum Disorder in Children with Facial Images and Analysis of Ethnoracial Factors in Model Development and Application

Autism spectrum disorder (ASD) is a developmental disability that can cause significant social, communication, and behavioral challenges. Early intervention for children with ASD can help to improve their intellectual ability and reduces autistic symptoms. Multiple clinical researches have suggested that facial phenotypic differences exist between ASD children and typically developing (TD) children. In this research, we propose a practical ASD screening solution using facial images through applying VGG16 transfer learning-based deep learning to a unique ASD dataset of clinically diagnosed children that we collected. Our model produced a 95% classification accuracy and 0.95 F1-score. The only other reported study using facial images to detect ASD was based on the Kaggle ASD Facial Image Dataset, which is an internet search-produced, low-quality, and low-fidelity dataset. Our results support the clinical findings of facial feature differences between children with ASD and TD children. The high F1-score achieved indicates that it is viable to use deep learning models to screen children with ASD. We concluded that the racial and ethnic-related factors in deep-learning based ASD screening with facial images are critical to solution viability and accuracy.

Organizing the Methodological Toolbox: Lessons Learned From Implementing Developmental Methods Online

Adapting studies typically run in the lab, preschool, or museum to online data collection presents a variety of challenges. The solutions to those challenges depend heavily on the specific questions pursued, the methods used, and the constraints imposed by available technology. We present a partial sample of solutions, discussing approaches we have developed for adapting studies targeting a range of different developmental populations, from infants to school-aged children, and utilizing various online methods such as high-framerate video presentation, having participants interact with a display on their own computer, having the experimenter interact with both the participant and an actor, recording free-play with physical objects, recording infant looking times both offline and live, and more. We also raise issues and solutions regarding recruitment and representativeness in online samples. By identifying the concrete needs of a given approach, tools that meet each of those individual needs, and interfaces between those tools, we have been able to implement many (but not all) of our studies using online data collection during the COVID-19 pandemic. This systematic review aligning available tools and approaches with different methods can inform the design of future studies, in and outside of the lab.