Clustering Hidden Markov Models With Variational Bayesian Hierarchical EM

Spatiotemporal Eye Movement Dynamics Reveal Altered Face Prioritization in Early Visual Processing Among Autistic Children

BACKGROUND

Reduced social attention-looking at faces-is one of the most common manifestations of social difficulty in autism that is central to social development. Although reduced social attention is well characterized in autism, qualitative differences in how social attention unfolds across time remains unknown.

METHODS

We used a computational modeling (i.e., hidden Markov modeling) approach to assess and compare the spatiotemporal dynamics of social attention in a large, well-characterized sample of children with autism (n = 280) and neurotypical children (n = 119) (ages 6-11) who completed 3 social eye-tracking assays at 3 longitudinal time points (baseline, 6 weeks, 24 weeks).

RESULTS

Our analysis supported the existence of 2 common eye movement patterns that emerged across 3 eye-tracking assays. A focused pattern was characterized by small face regions of interest, which had high a probability of capturing fixations early in visual processing. In contrast, an exploratory pattern was characterized by larger face regions of interest, with a lower initial probability of fixation and more nonsocial regions of interest. In the context of social perception, children with autism showed significantly more exploratory eye movement patterns than neurotypical children across all social perception assays and all 3 longitudinal time points. Eye movement patterns were associated with clinical features of autism, including adaptive function, face recognition, and autism symptom severity.

CONCLUSIONS

Decreased likelihood of precisely looking at faces early in social visual processing may be an important feature of autism that is associated with autism-related symptomology and may reflect less visual sensitivity to face information.

Understanding the Role of Eye Movement Pattern and Consistency in Isolated English Word Reading Through Hidden Markov Modeling

In isolated English word reading, readers have the optimal performance when their initial eye fixation is directed to the area between the beginning and word center, that is, the optimal viewing position (OVP). Thus, how well readers voluntarily direct eye gaze to this OVP during isolated word reading may be associated with reading performance. Using Eye Movement analysis with Hidden Markov Models, we discovered two representative eye movement patterns during lexical decisions through clustering, which focused at the OVP and the word center, respectively. Higher eye movement similarity to the OVP-focusing pattern predicted faster lexical decision time in addition to cognitive abilities and lexical knowledge. However, the OVP-focusing pattern was associated with longer isolated single letter naming time, suggesting conflicting visual abilities required for identifying isolated letters and multi-letter words. In contrast, in both word and pseudoword naming, although clustering did not reveal an OVP-focused pattern, higher consistency of the first fixation as measured in entropy predicted faster naming time in addition to cognitive abilities and lexical knowledge. Thus, developing a consistent eye movement pattern focusing on the OVP is essential for word orthographic processing and reading fluency. This finding has important implications for interventions for reading difficulties.

Understanding Human Cognition Through Computational Modeling

One important goal of cognitive science is to understand the mind in terms of its representational and computational capacities, where computational modeling plays an essential role in providing theoretical explanations and predictions of human behavior and mental phenomena. In my research, I have been using computational modeling, together with behavioral experiments and cognitive neuroscience methods, to investigate the information processing mechanisms underlying learning and visual cognition in terms of perceptual representation and attention strategy. In perceptual representation, I have used neural network models to understand how the split architecture in the human visual system influences visual cognition, and to examine perceptual representation development as the results of expertise. In attention strategy, I have developed the Eye Movement analysis with Hidden Markov Models method for quantifying eye movement pattern and consistency using both spatial and temporal information, which has led to novel findings across disciplines not discoverable using traditional methods. By integrating it with deep neural networks (DNN), I have developed DNN+HMM to account for eye movement strategy learning in human visual cognition. The understanding of the human mind through computational modeling also facilitates research on artificial intelligence's (AI) comparability with human cognition, which can in turn help explainable AI systems infer humans' belief on AI's operations and provide human-centered explanations to enhance human-AI interaction and mutual understanding. Together, these demonstrate the essential role of computational modeling methods in providing theoretical accounts of the human mind as well as its interaction with its environment and AI systems.

Modeling of nonlinear and nonstationary stochasticity for atomic ensembles

This paper addresses the problem of stochastic modeling of atomic ensembles under multi-source noise and makes the model interpretable. First, based on Itô's lemma and Allan variance analysis (ITÔ-AVAR), an approach is proposed to model nonstationary stochastic submodels of atomic ensembles. On this basis, the variance decomposition and nonlinear optimization algorithms are utilized to hybridize modeling atomic ensembles with nonlinear and nonstationary properties. Second, an Itô's lemma dynamic allan variance analysis (ITÔ-DAVAR) approach is developed for online modeling of atomic ensembles. Further, an atomic ensembles sensitivity enhancement scheme based on the proposed approach is given, which effectively promotes the progress of quantum instrument engineering. Finally, the proposed scheme are deployed in the optical pumping magnetometer and spin-exchange relaxation-free comagnetometer, respectively, while experimentally verifying the sensitivity of the spin-exchange relaxation-free comagnetometer reaches 5.36×10-6degs-1Hz-1/2.

Hidden Hypergraphs, Error-Correcting Codes, and Critical Learning in Hopfield Networks

In 1943, McCulloch and Pitts introduced a discrete recurrent neural network as a model for computation in brains. The work inspired breakthroughs such as the first computer design and the theory of finite automata. We focus on learning in Hopfield networks, a special case with symmetric weights and fixed-point attractor dynamics. Specifically, we explore minimum energy flow (MEF) as a scalable convex objective for determining network parameters. We catalog various properties of MEF, such as biological plausibility, and then compare to classical approaches in the theory of learning. Trained Hopfield networks can perform unsupervised clustering and define novel error-correcting coding schemes. They also efficiently find hidden structures (cliques) in graph theory. We extend this known connection from graphs to hypergraphs and discover n-node networks with robust storage of 2Ω(n1-ϵ) memories for any ϵ>0. In the case of graphs, we also determine a critical ratio of training samples at which networks generalize completely.

Eye movement analysis with hidden Markov models (EMHMM) with co-clustering

The eye movement analysis with hidden Markov models (EMHMM) method provides quantitative measures of individual differences in eye-movement pattern. However, it is limited to tasks where stimuli have the same feature layout (e.g., faces). Here we proposed to combine EMHMM with the data mining technique co-clustering to discover participant groups with consistent eye-movement patterns across stimuli for tasks involving stimuli with different feature layouts. Through applying this method to eye movements in scene perception, we discovered explorative (switching between the foreground and background information or different regions of interest) and focused (mainly looking at the foreground with less switching) eye-movement patterns among Asian participants. Higher similarity to the explorative pattern predicted better foreground object recognition performance, whereas higher similarity to the focused pattern was associated with better feature integration in the flanker task. These results have important implications for using eye tracking as a window into individual differences in cognitive abilities and styles. Thus, EMHMM with co-clustering provides quantitative assessments on eye-movement patterns across stimuli and tasks. It can be applied to many other real-life visual tasks, making a significant impact on the use of eye tracking to study cognitive behavior across disciplines.