The Einstellung effect in anagram problem solving: evidence from eye movements

The Einstellung effect is the counterintuitive finding that prior experience or domain-specific knowledge can under some circumstances interfere with problem solving performance. This effect has been demonstrated in several domains of expertise including medicine and chess. In the present study we explored this effect in the context of a simplified anagram problem solving task. Participants solved anagram problems while their eye movements were monitored. Each problem consisted of six letters: a central three-letter string whose letters were part of the solution word, and three additional individual letters. Participants were informed that one of the individual letters was a distractor letter and were asked to find a five-letter solution word. In order to examine the impact of stimulus familiarity on problem solving performance and eye movements, the central letter string was presented either as a familiar three-letter word, or the letters were rearranged to form a three-letter nonword. Replicating the classic Einstellung effect, overall performance was better for nonword than word trials. However, participants’ eye movements revealed a more complex pattern of both interference and facilitation as a function of the familiarity of the central letter string. Specifically, word trials resulted in shorter viewing times on the central letter string and longer viewing times on the individual letters than nonword trials. These findings suggest that while participants were better able to encode and maintain the meaningful word stimuli in working memory, they found it more challenging to integrate the individual letters into the central letter string when it was presented as a word.

Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective

Research on human and animal behavior has long emphasized its hierarchical structure-the divisibility of ongoing behavior into discrete tasks, which are comprised of subtask sequences, which in turn are built of simple actions. The hierarchical structure of behavior has also been of enduring interest within neuroscience, where it has been widely considered to reflect prefrontal cortical functions. In this paper, we reexamine behavioral hierarchy and its neural substrates from the point of view of recent developments in computational reinforcement learning. Specifically, we consider a set of approaches known collectively as hierarchical reinforcement learning, which extend the reinforcement learning paradigm by allowing the learning agent to aggregate actions into reusable subroutines or skills. A close look at the components of hierarchical reinforcement learning suggests how they might map onto neural structures, in particular regions within the dorsolateral and orbital prefrontal cortex. It also suggests specific ways in which hierarchical reinforcement learning might provide a complement to existing psychological models of hierarchically structured behavior. A particularly important question that hierarchical reinforcement learning brings to the fore is that of how learning identifies new action routines that are likely to provide useful building blocks in solving a wide range of future problems. Here and at many other points, hierarchical reinforcement learning offers an appealing framework for investigating the computational and neural underpinnings of hierarchically structured behavior.