Modeling accuracy as a function of response time with the generalized linear mixed effects model

MEG correlates of speech planning in simple vs. interactive picture naming in children and adults

The picture naming task is common both as a clinical task and as a method to study the neural bases of speech production in the healthy brain. However, this task is not reflective of most naturally occurring productions, which tend to happen within a context, typically in dialogue in response to someone else's production. How the brain basis of the classic "confrontation picture naming" task compares to the planning of utterances in dialogue is not known. Here we used magnetoencephalography (MEG) to measure neural activity associated with language production using the classic picture naming task as well as a minimal variant of the task, intended as more interactive or dialogue-like. We assessed how neural activity is affected by the interactive context in children, teenagers, and adults. The general pattern was that in adults, the interactive task elicited a robust sustained increase of activity in frontal and temporal cortices bilaterally, as compared to simple picture naming. This increase was present only in the left hemisphere in teenagers and was absent in children, who, in fact, showed the reverse effect. Thus our findings suggest a robustly bilateral neural basis for the coordination of interaction and a very slow developmental timeline for this network.

Lexico-Semantic Influence on Syntactic Processing: An Eye-Tracking Study with Spanish Relative Clauses

This paper investigates the interaction between lexicosemantic and syntactic information in sentence processing by examining the online comprehension of Spanish relative clauses (RCs) of both restrictive and non-restrictive types. A corpus study shows that, in Spanish, a RC may be introduced by different function words (called relativizers), which differ in lexical frequency, as well as semantic features. Based on these facts, an eye-tracking experiment was conducted with the aim of analyzing whether lexicosemantic information could influence sentence processing at the early stages. The results report an early influence of lexicosemantic information not only when activating a relativizer but also when integrating it within the syntactic structure. Additionally, the semantic role played by each RC type seems to constrain sentence processing at different regions. Our results favor an interactive view of language processing, according to which language comprehension is guided from the early stages by different kinds of linguistic information rather than syntactic information alone.

Early Form-Based Morphological Decomposition in Tagalog: MEG Evidence from Reduplication, Infixation, and Circumfixation

Neuro- and psycholinguistic experimentation supports the early decomposition of morphologically complex words within the ventral processing stream, which MEG has localized to the M170 response in the (left) visual word form area (VWFA). Decomposition into an exhaustive parse of visual morpheme forms extends beyond words like farmer to those imitating complexity (e.g., brother; Lewis et al., 2011), and to "unique" stems occurring in only one word but following the syntax and semantics of their affix (e.g., vulnerable; Gwilliams & Marantz, 2018). Evidence comes primarily from suffixation; other morphological processes have been under-investigated. This study explores circumfixation, infixation, and reduplication in Tagalog. In addition to investigating whether these are parsed like suffixation, we address an outstanding question concerning semantically empty morphemes. Some words in Tagalog resemble English winter as decomposition is not supported (wint-er); these apparently reduplicated pseudoreduplicates lack the syntactic and semantic features of reduplicated forms. However, unlike winter, these words exhibit phonological behavior predicted only if they involve a reduplicating morpheme. If these are decomposed, this provides evidence that words are analyzed as complex, like English vulnerable, when the grammar demands it. In a lexical decision task with MEG, we find that VWFA activity correlates with stem:word transition probability for circumfixed, infixed, and reduplicated words. Furthermore, a Bayesian analysis suggests that pseudoreduplicates with reduplicate-like phonology are also decomposed; other pseudoreduplicates are not. These findings are consistent with an interpretation that decomposition is modulated by phonology in addition to syntax and semantics.

The time course of normalizing speech variability in vowels

To achieve perceptual constancy, listeners utilize contextual cues to normalize speech variabilities in speakers. The present study tested the time course of this cognitive process with an event-related potential (ERP) experiment. The first neurophysiological evidence of speech normalization is observed in P2 (130-250 ms), which is functionally related to phonetic and phonological processes. Furthermore, the normalization process was found to ease lexical retrieval, as indexed by smaller N400 (350-470 ms) after larger P2. A cross-language vowel perception task was carried out to further specify whether normalization was processed in the phonetic and/or phonological stage(s). It was found that both phonetic and phonological cues in the speech context contributed to vowel normalization. The results suggest that vowel normalization in the speech context can be observed in the P2 time window and largely overlaps with phonetic and phonological processes.

Memory for emotional images across sleep versus wake in school-aged children

Sleep is important for emotional well-being, memory, and development in children. Regarding memory, sleep has been shown to advantage accuracy for declarative tasks but not procedural tasks. There is some evidence to suggest that sleep provides a relatively greater benefit for memory of negative emotional versus neutral images. However, the extent to which sleep benefits emotionally positive memories in children is not clear. This study assessed memory after nocturnal sleep versus daytime wake in a within-person design involving a sample of 40 children aged 7 to 14 years (M = 10.6 ± 1.9 years; 18 boys and 22 girls) for images of negative, neutral, and positive valences. Results show that after accounting for response time, memory accuracy overall was greater after sleep compared with equivalent time of wake and memory accuracy was greatest for positive images compared with both negative and neutral images. However, there was no difference between memory for negative images and that for neutral images in children, and there was no condition by valence interaction. Sleep characteristics as recorded using actigraphy, diary, and parent report were not predictive of memory performance after sleep when correcting for multiple comparisons. Overall, the results suggest that sleep may benefit memory in otherwise healthy children but that despite a bias toward memory for positive items over both negative and neutral items, there is not a relatively greater benefit for emotional versus neutral memory consolidation across sleep periods compared with wake periods.

The Language-specific Use of Fundamental Frequency Rise in Segmentation of an Artificial Language: Evidence from Listeners of Taiwanese Southern Min

Experience with native-language prosody encourages language-specific strategies for speech segmentation. Conflicting findings from previous research suggest that these strategies may not be abstracted away from the acoustic manifestation of prosodic features in the native speech. Using the artificial language learning paradigm, the current study explores this possibility in connection with listeners of a lexical tone language called Taiwanese Southern Min (TSM). In TSM, the only rising lexical tone occurs almost only on the final syllable of the language's tone sandhi domain and is phonetically associated with final lengthening. Based on these observations, Experiment I examined what constituted a sufficient finality cue for use by TSM listeners to support segmentation: (a) final fundamental frequency (F0) rise only; or (b) final F0 rise conjoined with final lengthening. The results showed that segmentation was inhibited by the former cue but facilitated by the latter. Experiment II showed that the facilitation cannot be attributed entirely to final lengthening, as a null effect was found when final lengthening was the sole prosodic cue to segmentation. It is thus assumed that acoustic details as fine-grained as the lengthening of the rising tone are involved in the modulation of the segmentation strategy whereby TSM listeners perceive F0 rise as signaling finality. The inhibitory effect of final F0 rise alone found in Experiment I motivated Experiment III, which revealed that initial F0 rise in the absence of lengthening cues improved TSM listeners' segmentation. It is speculated that such use of initial F0 rise might reflect a cross-linguistic segmentation solution.

How Computational Modeling Can Force Theory Building in Psychological Science

Psychology endeavors to develop theories of human capacities and behaviors on the basis of a variety of methodologies and dependent measures. We argue that one of the most divisive factors in psychological science is whether researchers choose to use computational modeling of theories (over and above data) during the scientific-inference process. Modeling is undervalued yet holds promise for advancing psychological science. The inherent demands of computational modeling guide us toward better science by forcing us to conceptually analyze, specify, and formalize intuitions that otherwise remain unexamined-what we dub open theory. Constraining our inference process through modeling enables us to build explanatory and predictive theories. Here, we present scientific inference in psychology as a path function in which each step shapes the next. Computational modeling can constrain these steps, thus advancing scientific inference over and above the stewardship of experimental practice (e.g., preregistration). If psychology continues to eschew computational modeling, we predict more replicability crises and persistent failure at coherent theory building. This is because without formal modeling we lack open and transparent theorizing. We also explain how to formalize, specify, and implement a computational model, emphasizing that the advantages of modeling can be achieved by anyone with benefit to all.

Quantifying the benefits of using decision models with response time and accuracy data

Response time and accuracy are fundamental measures of behavioral science, but discerning participants' underlying abilities can be masked by speed-accuracy trade-offs (SATOs). SATOs are often inadequately addressed in experiment analyses which focus on a single variable or which involve a suboptimal analytic correction. Models of decision-making, such as the drift diffusion model (DDM), provide a principled account of the decision-making process, allowing the recovery of SATO-unconfounded decision parameters from observed behavioral variables. For plausible parameters of a typical between-groups experiment, we simulate experimental data, for both real and null group differences in participants' ability to discriminate stimuli (represented by differences in the drift rate parameter of the DDM used to generate the simulated data), for both systematic and null SATOs. We then use the DDM to fit the generated data. This allows the direct comparison of the specificity and sensitivity for testing of group differences of different measures (accuracy, reaction time, and the drift rate from the model fitting). Our purpose here is not to make a theoretical innovation in decision modeling, but to use established decision models to demonstrate and quantify the benefits of decision modeling for experimentalists. We show, in terms of reduction of required sample size, how decision modeling can allow dramatically more efficient data collection for set statistical power; we confirm and depict the non-linear speed-accuracy relation; and we show how accuracy can be a more sensitive measure than response time given decision parameters which reasonably reflect a typical experiment.

Speed-Accuracy Tradeoffs in Brain and Behavior: Testing the Independence of P300 and N400 Related Processes in Behavioral Responses to Sentence Categorization

Although the N400 was originally discovered in a paradigm designed to elicit a P300 (Kutas and Hillyard, 1980), its relationship with the P300 and how both overlapping event-related potentials (ERPs) determine behavioral profiles is still elusive. Here we conducted an ERP (N = 20) and a multiple-response speed-accuracy tradeoff (SAT) experiment (N = 16) on distinct participant samples using an antonym paradigm (The opposite of black is white/nice/yellow with acceptability judgment). We hypothesized that SAT profiles incorporate processes of task-related decision-making (P300) and stimulus-related expectation violation (N400). We replicated previous ERP results (Roehm et al., 2007): in the correct condition (white), the expected target elicits a P300, while both expectation violations engender an N400 [reduced for related (yellow) vs. unrelated targets (nice)]. Using multivariate Bayesian mixed-effects models, we modeled the P300 and N400 responses simultaneously and found that correlation between residuals and subject-level random effects of each response window was minimal, suggesting that the components are largely independent. For the SAT data, we found that antonyms and unrelated targets had a similar slope (rate of increase in accuracy over time) and an asymptote at ceiling, while related targets showed both a lower slope and a lower asymptote, reaching only approximately 80% accuracy. Using a GLMM-based approach (Davidson and Martin, 2013), we modeled these dynamics using response time and condition as predictors. Replacing the predictor for condition with the averaged P300 and N400 amplitudes from the ERP experiment, we achieved identical model performance. We then examined the piecewise contribution of the P300 and N400 amplitudes with partial effects (see Hohenstein and Kliegl, 2015). Unsurprisingly, the P300 amplitude was the strongest contributor to the SAT-curve in the antonym condition and the N400 was the strongest contributor in the unrelated condition. In brief, this is the first demonstration of how overlapping ERP responses in one sample of participants predict behavioral SAT profiles of another sample. The P300 and N400 reflect two independent but interacting processes and the competition between these processes is reflected differently in behavioral parameters of speed and accuracy.

Cross-modal correspondences in sine wave: Speech versus non-speech modes

The present study aimed to investigate whether or not the so-called "bouba-kiki" effect is mediated by speech-specific representations. Sine-wave versions of naturally produced pseudowords were used as auditory stimuli in an implicit association task (IAT) and an explicit cross-modal matching (CMM) task to examine cross-modal shape-sound correspondences. A group of participants trained to hear the sine-wave stimuli as speech was compared to a group that heard them as non-speech sounds. Sound-shape correspondence effects were observed in both groups and tasks, indicating that speech-specific processing is not fundamental to the "bouba-kiki" phenomenon. Effects were similar across groups in the IAT, while in the CMM task the speech-mode group showed a stronger effect compared with the non-speech group. This indicates that, while both tasks reflect auditory-visual associations, only the CMM task is additionally sensitive to associations involving speech-specific representations.

Does the visual attention span play a role in the morphological processing of orthographic stimuli?

We investigated whether the link between visual attention (VA) span and reading is modulated by the presence of morphemes. Second and fourth grade children, with Basque as their first language, named morphologically complex and simple words and pseudowords, and performed a task measuring VA span. The influence of VA span skills on reading was modulated by the presence of morphemes in naming speed measures. In addition, fourth grade children with a larger VA span showed larger lexicality effects (pseudoword-word reading times) only for morphologically simple stimuli. Results are interpreted as support for the notion that both transparency and morphological complexity are important factors modulating the impact of VA span skills on reading.

Retrieval cues and syntactic ambiguity resolution: Speed-accuracy tradeoff evidence

Successful language comprehension often involves coping with lexical and syntactic ambiguity, and sometimes, recovering from misanalysis of the input. Syntactic ambiguity resolution has been shown throughout the literature to result in increases reaction time compared to unambiguous sentences, a fact which has shaped debates about architectures and mechanisms in sentence processing. However, increased reaction time can be caused either by a decrease in true processing speed, or by a decrease in the quality or quantity of information needed to reach criterion when making a response. Thus, increased reaction time to syntactic ambiguity could reflect differences in representational quality or multiple reanalysis attempts, or both. Current cue-based accounts of sentence processing predict that cues at the point where misanalysis becomes apparent (e.g., onset of second verb) may aid in reanalysis (e.g., retrieval of the correct subject). We used the speed-accuracy tradeoff procedure (SAT) to orthogonally derive estimates of processing speed and accuracy. We manipulated ambiguity and the semantic similarity between a disambiguating verb and the nouns already present in the sentence. Estimates of processing speed (SAT rate) indicated that, on average, ambiguous conditions took 250ms longer to interpret than unambiguous controls, demonstrating that reanalysis does increase veridical processing time. No interaction between cue diagnosticity and ambiguity was observed on speed or accuracy, but verbs more strongly related to the correct subject increased accuracy, regardless of ambiguity. These findings are consistent with a language processing architecture where cue-driven retrieval operations give rise to interpretation, and wherein diagnostic cues aid retrieval, regardless of parsing difficulty or structural uncertainty.

Application of Linear Mixed-Effects Models in Human Neuroscience Research: A Comparison with Pearson Correlation in Two Auditory Electrophysiology Studies

Neurophysiological studies are often designed to examine relationships between measures from different testing conditions, time points, or analysis techniques within the same group of participants. Appropriate statistical techniques that can take into account repeated measures and multivariate predictor variables are integral and essential to successful data analysis and interpretation. This work implements and compares conventional Pearson correlations and linear mixed-effects (LME) regression models using data from two recently published auditory electrophysiology studies. For the specific research questions in both studies, the Pearson correlation test is inappropriate for determining strengths between the behavioral responses for speech-in-noise recognition and the multiple neurophysiological measures as the neural responses across listening conditions were simply treated as independent measures. In contrast, the LME models allow a systematic approach to incorporate both fixed-effect and random-effect terms to deal with the categorical grouping factor of listening conditions, between-subject baseline differences in the multiple measures, and the correlational structure among the predictor variables. Together, the comparative data demonstrate the advantages as well as the necessity to apply mixed-effects models to properly account for the built-in relationships among the multiple predictor variables, which has important implications for proper statistical modeling and interpretation of human behavior in terms of neural correlates and biomarkers.

Language Processing as Cue Integration: Grounding the Psychology of Language in Perception and Neurophysiology

I argue that cue integration, a psychophysiological mechanism from vision and multisensory perception, offers a computational linking hypothesis between psycholinguistic theory and neurobiological models of language. I propose that this mechanism, which incorporates probabilistic estimates of a cue's reliability, might function in language processing from the perception of a phoneme to the comprehension of a phrase structure. I briefly consider the implications of the cue integration hypothesis for an integrated theory of language that includes acquisition, production, dialogue and bilingualism, while grounding the hypothesis in canonical neural computation.