Functional-anatomic study of episodic retrieval using fMRI. I. Retrieval effort versus retrieval success

Event-related functional imaging and episodic memory

Electrical and haemodynamic measures of neural activity can be time-locked to an event-of-interest, such as the presentation of a stimulus or a behavioural response. Both of these measures can be employed in studies where the aim is to elucidate the relationship between neural activity and cognitive processes. This review highlights a number of considerations that arise when these techniques are employed in pursuit of this goal, with a particular emphasis on functional imaging studies of retrieval from episodic memory. The review includes: a discussion of some limitations that each technique imposes at the stage of experimental design, consideration of the relative strengths and weaknesses of each technique, a commentary on assumptions that are common to both, and a brief review of the ways in which these techniques can be extended in order to index two distinct classes of cognitive operations that have correspondingly distinct neural signatures.

Anterior medial temporal lobe activation during attempted retrieval of encoded visuospatial scenes: an event-related fMRI study

Various studies have shown that the medial temporal lobe (MTL), which consists of the hippocampus and parahippocampal gyrus, is important for episodic memory. Earlier fMRI studies substantiated this role by showing activation upon encoding of visuospatial scenes. In this study we used event-related fMRI to study whether the cognitive process of retrieval of visuospatial scenes, tested with the use of a recognition paradigm, also activates the MTL. Nine subjects (mean age 24 years) were presented previously studied color pictures (old) and pictures they had never seen before (new) in a mixed trial design. Data analysis allowed calculation of the fMRI response of correct judgments on new pictures, old pictures, and false judgments. Since we used previously encoded color pictures as old stimuli, we also included an encoding paradigm in the current set of experiments. This allowed us to compare encoding and recognition activation in the MTL of exactly the same pictures in the same subjects. Correct judgments on new pictures showed an increased activation in the anterior parahippocampus bilaterally and the right anterior hippocampus compared to judgments on old pictures in the recognition experiment. The former judgments took significantly longer, indicating that retrieval of successfully stored information is less demanding than the effort to retrieve nonencoded information. A comparison of the two experimental data sets showed evidence for a functional segregation of encoding and retrieving color pictures. We conclude that the left posterior parahippocampal gyrus responds during encoding, while on the other hand the left anterior parahippocampal gyrus and the right anterior hippocampus were more strongly involved in retrieval.

Effect of schizophrenia on frontotemporal activity during word encoding and recognition: a PET cerebral blood flow study

OBJECTIVE

Neuropsychological studies have shown that deficits in verbal episodic memory in schizophrenia occur primarily during encoding and retrieval stages of information processing. The current study used positron emission tomography to examine the effect of schizophrenia on change in cerebral blood flow (CBF) during these memory stages.

METHOD

CBF was measured in 23 healthy comparison subjects and 23 patients with schizophrenia during four conditions: resting baseline, motor baseline, word encoding, and word recognition. The motor baseline was used as a reference that was subtracted from encoding and recognition conditions by using statistical parametric mapping.

RESULTS

Patients' performance was similar to that of healthy comparison subjects. During word encoding, patients showed reduced activation of left prefrontal and superior temporal regions. Reduced left prefrontal activation in patients was also seen during word recognition, and additional differences were found in the left anterior cingulate, left mesial temporal lobe, and right thalamus. Although patients' performance was similar to that of healthy comparison subjects, left inferior prefrontal activation was associated with better performance only in the comparison subjects.

CONCLUSIONS

Left frontotemporal activation during episodic encoding and retrieval, which is associated with better recognition in healthy people, is disrupted in schizophrenia despite relatively intact recognition performance and right prefrontal function. This may reflect impaired strategic use of semantic information to organize encoding and facilitate retrieval.

Direct comparison of prefrontal cortex regions engaged by working and long-term memory tasks

Neuroimaging studies have suggested the involvement of ventrolateral, dorsolateral, and frontopolar prefrontal cortex (PFC) regions in both working (WM) and long-term memory (LTM). The current study used functional magnetic resonance imaging (fMRI) to directly compare whether these PFC regions show selective activation associated with one memory domain. In a within-subjects design, subjects performed the n-back WM task (two-back condition) as well as LTM encoding (intentional memorization) and retrieval (yes-no recognition) tasks. Additionally, each task was performed with two different types of stimulus materials (familiar words, unfamiliar faces) in order to determine the influence of material-type vs task-type. A bilateral region of dorsolateral PFC (DL-PFC; BA 46/9) was found to be selectively activated during the two-back condition, consistent with a hypothesized role for this region in active maintenance and/or manipulation of information in WM. Left frontopolar PFC (FP-PFC) was also found to be selectively engaged during the two-back. Although FP-PFC activity has been previously associated with retrieval from LTM, no frontopolar regions were found to be selectively engaged by retrieval. Finally, lateralized ventrolateral PFC (VL-PFC) regions were found to be selectively engaged by material-type, but uninfluenced by task-type. These results highlight the importance of examining PFC activity across multiple memory domains, both for functionally differentiating PFC regions (e.g., task-selectivity vs material-selectivity in DL-PFC and VL-PFC) and for testing the applicability of memory domain-specific theories (e.g., FP-PFC in LTM retrieval).

A functional magnetic resonance imaging study of auditory mismatch in schizophrenia

OBJECTIVE

Previous research has noted functional and structural temporal lobe abnormalities in schizophrenia that relate to symptoms such as auditory hallucinations and thought disorder. The goal of the study was to determine whether the functional abnormalities are present in schizophrenia at early stages of auditory processing.

METHOD

Functional magnetic resonance imaging activity was examined during the presentation of the mismatch stimuli, which are deviant tones embedded in a series of standard tones. The mismatch stimuli are used to elicit the mismatch negativity, an early auditory event-related potential. Ten patients with schizophrenia and 10 comparison subjects were presented the mismatch stimuli condition and a control condition in which only one tone was presented repeatedly.

RESULTS

The superior temporal gyrus showed the most prevalent and consistent activation. The superior temporal gyrus showed less activation in the schizophrenic subjects than in the comparison subjects only during the mismatch stimuli condition.

CONCLUSIONS

This result is consistent with those of mismatch negativity event-related potential studies and suggests that early auditory processing is abnormal in chronic schizophrenia.

Parahippocampal Activation during Successful Recognition of Words: A Self-Paced Event-Related fMRI Study

In this study, we investigated retrieval from verbal episodic memory using a self-paced event-related fMRI paradigm, similar to the designs typically used in behavioral studies of memory function. We tested the hypothesis that the medial temporal lobe (MTL) is involved in the actual recovery of verbal information (retrieval success) rather than in the attempt to retrieve information (retrieval attempt). To this end, we used a verbal recognition task, distinguishing correctly recognized words, correctly rejected words, and a low-level baseline condition. Directly contrasting correct recognition with correct rejection of words, we found activation in the left fusiform/parahippocampal gyrus, indicating that this region has a distinct role in the successful retrieval of verbal information. Furthermore, our results were in agreement with those of previous imaging studies that compared a fixed-paced verbal recognition task to a baseline condition, showing activation in bilateral inferior frontal cortex, left dorsolateral prefrontal cortex, left anterior insular cortex, and anterior cingulate. This demonstrates the applicability of a self-paced event-related design within imaging studies of memory function.

Control of semantic interference in episodic memory retrieval is associated with an anterior cingulate-prefrontal activation pattern

Prefrontal activation is a consistent finding in functional neuroimaging studies of episodic memory retrieval. In the present study we aimed at a further analysis of prefrontal neural systems involved in the executive control of context-specific properties in episodic memory retrieval using an event-related fMRI design. Nine subjects were asked to learn two 20-item word lists that consisted of concrete nouns assigned to four semantic categories. Ten items of both word lists referred to the same semantic category. Subjects were instructed to determine whether nouns displayed in random order corresponded to the first 20-item target list. The interference evoked by the retrieval of semantically related items of the second list resulted in significantly longer reaction times compared to the noninterference condition. Contrasting the interference against the noninterference retrieval condition demonstrated an activation pattern that comprised a right anterior cingulate and frontal opercular area and a left-lateralized dorsolateral prefrontal region. Trial averaged time series revealed that the PFC areas were selectively activated at the interference condition and did not respond to the familiarity of learned words. These findings suggest a functionally separable role of prefrontal cortical areas mediating processes associated with the executive control of interfering context information in episodic memory retrieval.

Encoding Processes during Retrieval Tasks

Episodic memory encoding is pervasive across many kinds of task and often arises as a secondary processing effect in tasks that do not require intentional memorization. To illustrate the pervasive nature of information processing that leads to epeisodic encoding, a form of incidental encoding was explored based on the “Testing” phenomenon: The incidental-encoding task was an episodic memory retrieval task. Behavioral data showed that performing a memory retrieval task was as effedctive as intentional instructions at promoting episodic encoding. During fMRI imaging, subjedcts veiewed old and new words adn indicated whether they remembered them. Relevant to encoding, the fate of the new words was examined using a second, surprise test of recognition after the imaging session, fMRI analysis of those new words that were later remembered revealed greater activity in left frontal regions than those that were later forgotten-the same pattern of results as previously observed for traditional incidental and intentional episodic encoding tasks. This finding may offer a partial explanation for why repeated testing improves memory performance. Furthermore, the observation of correlates of episodic memory encoding during retrieval tasks challenges some interpretations that aris from direct comparisons between: encoding tasks and “retrieval tasks” in imaging data. Encoding processes and their neural correlates may arise in many tasks, even those nominally labeled as retrieval tasks by the experimenter.

Brain potentials reflect behavioral differences in true and false recognition

People often falsely recognize nonstudied lures that are semantically similar to previously studied words. Behavioral research suggests that such false recognition is based on high semantic overlap between studied items and lures that yield a feeling of familiarity, whereas true recognition is more often associated with the recollection of details. Despite this behavioral evidence for differences between true and false recognition, research measuring brain activity (PET, fMRI, ERP) has not clearly differentiated corresponding differences in brain activity. A median split was used to separate subjects into Good and Poor performers based on their discrimination of studied targets from similar lures. Only Good performers showed late (1000--1500 msec), right frontal event-related brain potentials (ERPs) that were more positive for targets and lures compared with new items. The right frontal differences are interpreted as reflecting postretrieval evaluation processes that were more likely to be engaged by Good than Poor performers. Both Good and Poor performers showed a parietal ERP old/new effect (400--800 msec), but only Poor performers showed a parietal old/lure difference. These results are consistent with the view that the parietal and frontal ERP old/new effects reflect dissociable processes related to recollection.

Dissociating state and item components of recognition memory using fMRI

Cognitive functions such as memory retrieval involve a combination of state- and item-related processes. State-related processes are sustained throughout a task (e.g., "retrieval mode" associated with ongoing goals), whereas item-related processes are transient and allied to individual stimuli (e.g., "retrieval success" associated with the recovery of information from memory). The present study employed a mixed "blocked and event-related" experimental design to identify neural mechanisms that support state- and item-related processes during a recognition memory task. Subjects alternated between blocks of fixation and recognition memory (discriminating between old and new words). Critically, event-related procedures were embedded within the recognition blocks, including the jittering of sequential trials. This design ensures that the temporal profiles of state- and item-related activity differ and consequently renders them separable; without this procedure item-related activity would summate to produce a state-like response. Results suggest three classes of brain region support recognition memory, exhibiting: (1) predominantly transient activity (including regions in medial parietal, lateral parietal, and anterior left frontal cortex) reflecting item-related processing associated with "retrieval success," (2) predominantly sustained activity (including decreased activity in bilateral parahippocampal cortex) reflecting state-related processing associated with "retrieval mode," (3) concurrent sustained and transient activity (including regions in left middle frontal gyrus, bilateral frontal operculum, and medial frontal gyrus), reflecting a combination of state- and item-related processing. The present findings support the idea that recognition memory tasks are dependent upon a combination of state- and item-related processes that have dissociable neural correlates identifiable using fMRI. Moreover, the mixed "blocked and event-related" design employed here provides a general procedure for separating state- and item-related processes.

Event related brain potentials and illusory memories: the effects of differential encoding

This study investigates event related potentials (ERP) elicited by true and false recognition using words from different semantic categories. In Experiment 1, ERPs for true and false recognition were more positive than for correctly rejected NEW words starting around 300 ms after test word presentation (old/new ERP effects). ERP waveforms for true and false recognition revealed equal early (300-500 ms) fronto-medial old/new ERP effects, reflecting similar familiarity processes, but smaller parietal old/new ERP effects (500-700 ms) for false relative to true recognition, suggesting less active recollection. Interestingly, a subsequent performance based group comparison showed equivalent old/new ERP effects for true and false recognition for participants with high rates of false recognition. In contrast, false recognition failed to elicit an old/new ERP effect in a group with low false recognition rates. To examine whether this between group difference was driven by the differential use of information that studied words and semantically related non studied test words (LURE) have in common (conceptual similarity), we manipulated encoding strategy in Experiment 2. When encoding focused on conceptual similarity, comparable ERP-effects for true and false recognition were obtained, suggesting that both forms of recognition were equally based on familiarity and recollection processes. Conversely, when encoding was focused on item specific features, differences in brain activity for true and false recognition were obtained. The ERP data indicate that, in addition to the false recognition rate, strategic processes during encoding, such as processing conceptual features, are an important factor in determining electrophysiological differences between true and false recognition.

Confidence in recognition memory for words: dissociating right prefrontal roles in episodic retrieval

We used event-related functional magnetic resonance imaging (efMRI) to investigate brain regions showing differential responses as a function of confidence in an episodic word recognition task. Twelve healthy volunteers indicated whether their old-new judgments were made with high or low confidence. Hemodynamic responses associated with each judgment were modeled with an "early" and a "late" response function. As predicted by the monitoring hypothesis generated from a previous recognition study [Henson, R. N. A., Rugg, M. D., Shallice, T., Josephs, O., & Dolan, R. J. (1999a). Recollection and familiarity in recognition memory: An event-related fMRI study. Journal of Neuroscience, 19, 3962-3972], a right dorsolateral prefrontal region showed a greater response to correct low- versus correct high-confidence judgements. Several regions, including the precuneus, posterior cingulate, and left lateral parietal cortex, showed greater responses to correct old than correct new judgements. The anterior left and right prefrontal regions also showed an old-new difference, but for these regions the difference emerged relatively later in time. These results further support the proposal that different subregions of the prefrontal cortex subserve different functions during episodic retrieval. These functions are discussed in relation to a monitoring process, which operates when familiarity levels are close to response criterion and is associated with nonconfident judgements, and a recollective process, which is associated with the confident recognition of old words.

Remembering episodes: a selective role for the hippocampus during retrieval

Some memories are linked to a specific time and place, allowing one to re-experience the original event, whereas others are accompanied only by a feeling of familiarity. To uncover the distinct neural bases for these two types of memory, we measured brain activity during memory retrieval using event-related functional magnetic resonance imaging. We show that activity in the hippocampus increased only when retrieval was accompanied by conscious recollection of the learning episode. Hippocampal activity did not increase for items recognized based on familiarity or for unrecognized items. These results indicate that the hippocampus selectively supports the retrieval of episodic memories.

Response-related fMRI analysis during encoding and retrieval revealed differences in cerebral activation by retrieval success

The aim of the study was to identify cerebral activation associated with sufficient or insufficient encoding, and with correct or false recognition. Fourteen volunteers performed two paradigms: explicit learning of words; and later retrieval of previously presented words. Items were classified according to the subjects' recognition performance. Echo-planar MRI of blood-oxygen-level-dependent signal changes was performed during encoding and retrieval. Response-related fMRI-analysis was used to compare activation associated with the subjects' retrieval success. During encoding, there was a trend towards increased activation of the left medial cingulate gyrus and of the right fusiform gyrus for later hits (correctly identified, learned target words) in comparison with misses (non-identified targets). During recognition, signal intensities associated with false alarms (falsely identified distractors) were significantly higher in left and right extrastriate cortex than those associated with hits, misses and correct rejections of distractors. Activation in the anterior cingulate gyrus during retrieval was related to reaction time and might be associated with the preparation or performance of motor response. Increased activation during false alarms might reflect a source-monitoring deficit or an increased subjective familiarity with distractors that have been most intensively processed in extrastriate visual cortex.

Methodological issues in pediatric neuroimaging

The emergence of new technologies to study brain function in vivo has resulted in an explosion of interest in cognitive neuroscience within the last ten years. While most research in functional neuroimaging has been geared toward adult normal volunteers, the development of functional magnetic resonance imaging (fMRI) has made it possible to study neural development in normal children, as well as those with developmental disorders. This technology provides an unprecedented opportunity to expand our knowledge of brain function throughout childhood. A variety of technological, experimental, and practical difficulties are amplified when imaging children. This paper reviews some of the more challenging theoretical and practical concerns and provides suggestions for their management.

Event-related potential (ERP) studies of memory encoding and retrieval: a selective review

As event-related brain potential (ERP) researchers have increased the number of recording sites, they have gained further insights into the electrical activity in the neural networks underlying explicit memory. A review of the results of such ERP mapping studies suggests that there is good correspondence between ERP results and those from brain imaging studies that map hemodynamic changes. This concordance is important because the combination of the high temporal resolution of ERPs with the high spatial resolution of hemodynamic imaging methods will provide a greatly increased understanding of the spatio-temporal dynamics of the brain networks that encode and retrieve explicit memories.

Decomposing memory: functional assignments and brain traffic in paired word associate learning

The recent covariance structural equation model for word-pair associate encoding and retrieval (Krause, Horwitz, Taylor, Schmidt, Mottaghy, Halsband et al., 1998; Krause, Horwitz, Taylor, Schmidt, Mottaghy, Herzog et al., 1999) is analysed to deduce possible functional assignments of the various brain modules used by subjects in solving the task. Specific processing aspects are considered, in particular, that of long-term working memory sites and how they are coupled to buffer working memory sites to enable deposition and manipulation of remembered associates. The new concept of 'brain traffic' is introduced as an aid to the assessment of how important are various brain modules. A set of functional assignments is produced for the relevant modules.

Network analysis of brain activations in working memory: behavior and age relationships

Forty-six middle-aged female subjects were scanned using functional Magnetic Resonance Imaging (fMRI) during performance of three distinct stages of a working memory task-encoding, rehearsal, and recognition-for both printed pseudowords and visual forms. An expanse of areas, involving the inferior frontal, parietal, and extrastriate cortex, was active in response to stimuli during both the encoding and recognition periods. Additional increases during memory recognition were seen in right prefrontal regions, replicating a now-common finding [for reviews, see Fletcher et al. (1997) Trends Neurosci 20:213-218; MacLeod et al. (1998) NeuroImage 7:41-48], and broadly supporting the Hemispheric Encoding/Retrieval Asymmetry hypothesis [Tulving et al. (1994) Proc Natl Acad Sci USA 91:2016-2020]. Notably, this asymmetry was not qualified by the type of material being processed. A few sites demonstrated higher activity levels during the rehearsal period, in the absence of any new stimuli, including the medial extrastriate, precuneus, and the medial temporal lobe. Further analyses examined relationships among subjects' brain activations, age, and behavioral scores on working memory tests, acquired outside the scanner. Correlations between brain scores and behavior scores indicated that activations in a number of areas, mainly frontal, were associated with performance. A multivariate analysis, Partial Least Squares [McIntosh et al. (1996) NeuroImage 3:143-157, (1997) Hum Brain Map 5:323-327], was then used to extract component effects from this large set of univariate correlations. Results indicated that better memory performance outside the scanner was associated with higher activity at specific sites within the frontal and, additionally, the medial temporal lobes. Analysis of age effects revealed that younger subjects tended to activate more than older subjects in areas of extrastriate cortex, medial frontal cortex, and the right medial temporal lobe; older subjects tended to activate more than younger subjects in the insular cortex, right inferior temporal lobe, and right inferior frontal gyrus. These results extend recent reports indicating that these regions are specifically involved in the memory impairments seen with aging.

Neural correlates of episodic retrieval success

Episodic memory retrieval involves multiple component processes, including those that occur when information is correctly remembered (retrieval success). The present study employed rapid-presentation event-related functional MRI that allowed different trial types with short intertrial intervals to be sorted such that the hemodynamic response associated with retrieval success could be extracted. Specifically, in an old/new episodic recognition task, hit trials (correctly recognized old items) and correct rejection trials (correctly rejected new items) were directly compared. The comparison revealed a mostly left-lateralized set of brain regions. Differential activation was most robust in left lateral parietal cortex and medial parietal cortex. Additional regions of differential activation included left anterior prefrontal cortex at or near Brodmann area 10, anterior insula, thalamus, anterior cingulate cortex, frontal cortex along inferior frontal gyrus, premotor cortex, and presupplementary motor area. These results suggest that left frontal and parietal regions modulate activity based on the successful retrieval of information from episodic memory. We discuss these findings in the context of several recent investigations that provide converging results as well as prior studies that have failed to detect these changes.

Electrophysiological evidence for the modulation of retrieval orientation by depth of study processing

Event-related potentials (ERPs) were employed to investigate whether brain activity elicited by retrieval cues in a memory test varies according to the encoding task undertaken at study. Two recognition memory test blocks were administered, preceded, in one case, by a "shallow" study task (alphabetic judgement) and, in the other case, by a "deep" task (sentence generation). ERPs elicited by the new words in each test block differed, the ERPs elicited in the block following the shallow study task exhibiting the more positive-going waveforms. This finding was taken as evidence that subjects adopt different "retrieval sets" when attempting to retrieve items that had been encoded in terms of alphabetic versus semantic attributes. Differences between the ERPs elicited by correctly classified old and new words (old/new effects) also varied with encoding task. The effects for deeply studied words resembled those found in previous ERP studies of recognition memory, whereas old/new effects for shallowly studied words were confined to a late-onsetting, right frontal positivity. Together, the findings indicate that the depth of study processing influences two kinds of memory-related neural activity, associated with memory search operations, and the processing of retrieved information, respectively.

Segregating semantic and syntactic aspects of processing in the human brain: an fMRI investigation of different word types

The processing of single words that varied in their semantic (concrete/abstract word) and syntactic (content/function word) status was investigated under different task demands (semantic/ syntactic task) in an event-related functional magnetic resonance imaging experiment. Task demands to a large degree determined which subparts of the neuronal network supporting word processing were activated. Semantic task demands selectively activated the left pars triangularis of the inferior frontal gyrus (BA 45) and the posterior part of the left middle/superior temporal gyrus (BA 21/22/37). In contrast, syntactic processing requirements led to an increased activation in the inferior tip of the left frontal operculum (BA 44) and the cortex lining the junction of the inferior frontal and inferior precentral sulcus (BA 44/6). Moreover, for these latter areas a word class by concreteness interaction was observed when a syntactic judgement was required. This interaction can be interpreted as a prototypicality effect: non-prototypical members of a word class, i.e. concrete function words and abstract content words, showed a larger activation than prototypical members, i.e. abstract function words and concrete content words. The combined data suggest that the activation pattern underlying word processing is predicted neither by syntactic class nor semantic concreteness but, rather, by task demands focusing either on semantic or syntactic aspects. Thus, our findings that semantic and syntactic aspects of processing are both functionally distinct and involve different subparts of the neuronal network underlying word processing support a domain-specific organization of the language system.

Functional asymmetry of human prefrontal cortex: encoding and retrieval of verbally and nonverbally coded information

There are several views about the organization of memory functions in the human prefrontal cortex. One view assumes a process-specific brain lateralization according to different memory subprocesses, that is, encoding and retrieval. An alternative view emphasizes content-specific lateralization of brain systems involved in memory processes. This study addresses this apparent inconsistency between process- and content-specific lateralization of brain activity by investigating the effects of verbal and nonverbal encoding on prefrontal activations during encoding and retrieval of environmental novel sounds using fMRI. An intentional memory task was applied in which subjects were required either to judge the sounds' loudness (nonverbal encoding task) or to indicate whether or not a sound can be verbally described (verbal encoding task). Retrieval processes were examined in a subsequent yes/no recognition test. In the study phase the right posterior dorsolateral prefrontal cortex (PFC) was activated in both tasks. During verbal encoding additional activation of the left dorsolateral PFC was obtained. Retrieval-related fMRI activity varied as a function of encoding task: For the nonverbal task we detected an activation focus in the right posterior dorsolateral PFC whereas an activation in the left dorsolateral PFC was observed for the verbal task. These findings indicate that the right dorsolateral PFC is engaged in encoding of auditory information irrespective of encoding task. The lateralization of PFC activity during retrieval was shown to depend on the availability of verbal codes, with left hemispheric involvement for verbally and right hemispheric activation for nonverbally coded information.

Retrieval processing and episodic memory

The emergence of brain imaging has had a major impact on research into the cognitive and neural bases of human memory. An area in which this impact has been particularly strong is retrieval processing - the processes engaged when attempting to retrieve information during a memory test. Several different classes of retrieval process - such as 'mode', 'effort' and 'success' - have been invoked to account for findings from neuroimaging studies of episodic retrieval. In this article we discuss how these different kinds of process, along with a fourth kind associated with 'retrieval orientation', can be investigated in brain imaging experiments. We then review studies of retrieval processing, and assess how well their designs match up to our proposed criteria for dissociating the neural correlates of different classes of retrieval process. We conclude that few studies have used designs that permit these different kinds of process to be independently identified, and that presently there is little evidence to indicate which kinds of processing can be fractionated in terms of their neural correlates.

Neural correlates of memory retrieval and evaluation

Results from recent neuroimaging studies have led to a controversy as to whether right or left prefrontal regions are relatively more important for episodic retrieval. To address this issue, we recorded event-related brain potentials during two recognition tests with identical stimuli but differing retrieval demands. In both tests, participants viewed a sequence of object drawings, half of which were identical to ones viewed earlier except for a change in size and half of which were new. Instructions were to discriminate between old and new objects (general test) or to additionally decide whether old objects were larger or smaller at study (specific test). Frontal brain potentials that were more positive during the specific than during the general test for both old and new objects were interpreted as neural correlates of the process by which specific attributes of test cues are compared with information retrieved from memory. Another ERP difference between the specific and general tests, which was observed for old objects only, had a left posterior scalp topography and was interpreted to reflect the reactivation of memories for studied objects. Frontal and posterior potentials thus reflected two memory processes important for accurate episodic retrieval. Furthermore, our findings suggest that both left and right prefrontal regions were engaged when demands to retrieve and evaluate perceptual information increased.

Time-dependent changes in learning audiovisual associations: a single-trial fMRI study

Functional imaging studies of learning and memory have primarily focused on stimulus material presented within a single modality (see review by Gabrieli, 1998, Annu. Rev. Psychol. 49: 87-115). In the present study we investigated mechanisms for learning material presented in visual and auditory modalities, using single-trial functional magnetic resonance imaging. We evaluated time-dependent learning effects under two conditions involving presentation of consistent (repeatedly paired in the same combination) or inconsistent (items presented randomly paired) pairs. We also evaluated time-dependent changes for bimodal (auditory and visual) presentations relative to a condition in which auditory stimuli were repeatedly presented alone. Using a time by condition analysis to compare neural responses to consistent versus inconsistent audiovisual pairs, we found significant time-dependent learning effects in medial parietal and right dorsolateral prefrontal cortices. In contrast, time-dependent effects were seen in left angular gyrus, bilateral anterior cingulate gyrus, and occipital areas bilaterally. A comparison of paired (bimodal) versus unpaired (unimodal) conditions was associated with time-dependent changes in posterior hippocampal and superior frontal regions for both consistent and inconsistent pairs. The results provide evidence that associative learning for stimuli presented in different sensory modalities is supported by neural mechanisms similar to those described for other kinds of memory processes. The involvement of posterior hippocampus and superior frontal gyrus in bimodal learning for both consistent and inconsistent pairs supports a putative function for these regions in associative learning independent of sensory modality.

The role of the right anterior prefrontal cortex in episodic retrieval

Regional brain activity was measured with H(2) (15)O PET while participants attempted to complete word-stem and word-fragment retrieval cues with previously studied words. The retrieval cue manipulation was employed to gain control over the monitoring operations associated with evaluating the episodic status of alternative cue completions. These operations were more constrained for fragments, which had fewer possible completions than each corresponding stem. In one condition (zero target), during the scanning interval none of the cues could be completed with studied items, whereas in another condition (high target), 80% of cues belonged to studied items. Relative to baseline tasks, right anterior prefrontal activity was greater for stems than for fragments in the zero target condition. The target density manipulation did not modulate right anterior prefrontal activity, but was associated with increased activity in right dorsolateral prefrontal cortex. These findings are consistent with the proposal that the right anterior prefrontal cortex supports monitoring operations during episodic retrieval tasks. In addition, the findings add to evidence suggesting that the dorsolateral and anterior right prefrontal cortex make functionally distinct contributions to episodic retrieval.

Prefrontal cortex and episodic memory retrieval mode

A multistudy analysis of positron emission tomography data identified three right prefrontal and two left prefrontal cortical sites, as well as a region in the anterior cingulate gyrus, where neuronal activity is correlated with the maintenance of episodic memory retrieval mode (REMO), a basic and necessary condition of remembering past experiences. The right prefrontal sites were near the frontal pole [Brodmann's area (BA) 10], frontal operculum (BA 47/45), and lateral dorsal area (BA 8/9). The two left prefrontal sites were homotopical with the right frontal pole and opercular sites. The same kinds of REMO sites were not observed in any other cerebral region. Many previous functional neuroimaging studies of episodic memory retrieval have reported activations near the frontal REMO sites identified here, although their function has not been clear. Many of these, too, probably have signaled their involvement in REMO. We propose that REMO activations largely if not entirely account for the frontal hemispheric asymmetry of retrieval as described by the original hemispheric encoding retrieval asymmetry model.

Acupuncture modulates the limbic system and subcortical gray structures of the human brain: evidence from fMRI studies in normal subjects

Acupuncture, an ancient therapeutic technique, is emerging as an important modality of complementary medicine in the United States. The use and efficacy of acupuncture treatment are not yet widely accepted in Western scientific and medical communities. Demonstration of regionally specific, quantifiable acupuncture effects on relevant structures of the human brain would facilitate acceptance and integration of this therapeutic modality into the practice of modern medicine. Research with animal models of acupuncture indicates that many of the beneficial effects may be mediated at the subcortical level in the brain. We used functional magnetic resonance imaging (fMRI) to investigate the effects of acupuncture in normal subjects and to provide a foundation for future studies on mechanisms of acupuncture action in therapeutic interventions. Acupuncture needle manipulation was performed at Large Intestine 4 (LI 4, Hegu) on the hand in 13 subjects [Stux, 1997]. Needle manipulation on either hand produced prominent decreases of fMRI signals in the nucleus accumbens, amygdala, hippocampus, parahippocampus, hypothalamus, ventral tegmental area, anterior cingulate gyrus (BA 24), caudate, putamen, temporal pole, and insula in all 11 subjects who experienced acupuncture sensation. In marked contrast, signal increases were observed primarily in the somatosensory cortex. The two subjects who experienced pain instead of acupuncture sensation exhibited signal increases instead of decreases in the anterior cingulate gyrus (BA 24), caudate, putamen, anterior thalamus, and posterior insula. Superficial tactile stimulation to the same area elicited signal increases in the somatosensory cortex as expected, but no signal decreases in the deep structures. These preliminary results suggest that acupuncture needle manipulation modulates the activity of the limbic system and subcortical structures. We hypothesize that modulation of subcortical structures may be an important mechanism by which acupuncture exerts its complex multisystem effects.

Memory retrieval under the control of the prefrontal cortex

Memory retrieval is a process wherein a distributed neural network reactivates the brain's representation of past experiences. Sensory long-term memory is represented among a population of neurones in the modality-specific posterior association cortex. The coded representation of memory can be retrieved by interactions of hierarchically different cortical areas along bottom-up and top-down anatomical connections. We examined the function of the prefrontal cortex in memory retrieval by two different approaches. Firstly, a meta-analysis of brain imaging studies revealed that the prefrontal cortex is reliably activated by memory retrieval in humans. Secondly, in order to determine the causal relationship between the prefrontal activations and memory retrieval, we designed a new experimental paradigm using posterior-split-brain monkeys. Following section of the splenium of the corpus callosum and the anterior commissure, visual stimulus-stimulus association learning within one hemisphere did not transfer to the other. Nevertheless, when a visual cue was presented to one hemisphere, the prefrontal cortex could instruct the contralateral hemisphere to retrieve the correct stimulus specified by the cue. These findings suggest that the prefrontal cortex can regulate the recall of long-term memory in the absence of bottom-up sensory inputs.

Functional MRI of cerebral activation during encoding and retrieval of words

The aims of the present study were to identify the cerebral structures associated with encoding and retrieval of verbal material. To circumvent the inherent disadvantages of the conventional block designs used in functional magnetic resonance imaging (MRI), an event-related design compared activation related to randomly intermixed old and new words during recognition. To support the validity of results, both nonparametric analyses in regions of interest (ROI) and statistical parametric mapping (SPM 96) were used. Twelve healthy volunteers, ages 22-35 years, performed three tasks: intentional encoding of words, recognition of old (previously learned) words, and discrimination between words and nonwords, a task to control for visual input and motor output during recognition. Echo-planar magnetic resonance imaging of blood-level, oxygen-dependent, task-related changes was used to compare cerebral activity under active and resting conditions as well as to detect event-related activity within blocks of trials. Comparable results were obtained following nonparametric statistical analysis of selected ROI and SPM. Encoding of words was associated with increased activity in the left inferior frontal gyrus, including Broca's area and in the left parietal association cortex. Event-related data analysis revealed activation of the right medial frontal gyrus, the right anterior cingulate gyrus, and parietal association cortices during recognition of previously presented words. In the lexical decision task, words in comparison with nonwords were associated with activation of the left parietal association cortex. The right medial frontal gyrus, the right anterior cingulate gyrus, and the right parietal association cortex are likely to be involved in episodic memory functions during recognition of previously presented verbal material. The comparison of event-related activation occurring within one trial block instead of among several trial blocks may significantly improve the performance of memory studies.

The role of the prefrontal cortex in recognition memory and memory for source: an fMRI study

We employed fMRI to index neural activity in prefrontal cortex during tests of recognition and source memory. At study, subjects were presented with words displayed either to the left or right of fixation, and, depending on the side, performed one of two orienting tasks. The test phase consisted of a sequence of three 10-word blocks, displayed in central vision. For one block, subjects performed recognition judgements on a mixture of two old and eight new words (low density recognition). For another block, recognition judgements were performed on a mixture of eight old and two new words (high density recognition). In the remaining block, also consisting of eight old and two new items, the requirement was to judge whether each word had been presented at study on the left or the right. Relative to the low density condition, high density recognition was associated with increased activity in right and, to a lesser extent, left, anterior prefrontal cortex (BA 10), replicating the findings of two previous PET studies. Right anterior prefrontal activity did not show any further increase during the source task. Instead, greater activity was found, relative to high density recognition, in left BA 10, left inferior frontal gyrus (BA 45/47), and bilateral opercular cortices (BA 45/47). The findings are inconsistent with the proposal that activation of right anterior prefrontal cortex during memory retrieval reflects "postretrieval" processing demands, such demands being considerably greater for judgments of source than recognition. The findings provide further evidence that the left prefrontal cortex plays a role in episodic memory retrieval when the task explicitly requires recovery of contextual as well as item information.

Set- and code-specific activation in frontal cortex: an fMRI study of encoding and retrieval of faces and words

The frontal cortex has been described as playing both "set-specific" and "code-specific" roles in human memory processing. Set specificity refers to the finding of goal-oriented differences in activation patterns (e.g., encoding relative to retrieval). Code specificity refers to the finding of different patterns of activation for different types of stimuli (e.g., verbal/nonverbal). Using a two (code: verbal, nonverbal) by two (set: encoding, retrieval) within-subjects design and fMRI, we explored the influence of type of code and mental set in two regions in the frontal cortex that have been previously shown to be involved in memory. A region in the dorsal extent of the inferior frontal gyrus (BA 6/44) demonstrated code-specific effects. Specifically, an interaction of material type with hemisphere was obtained, such that words produced predominantly left-lateralized activation, whereas unfamiliar faces elicited predominantly right-lateralized activation. A region of the right frontal polar cortex (in or near BA 10), which has been activated in many memory retrieval studies, showed set-specific activation in that it was more active during retrieval than encoding. These data demonstrate that distinct regions in the frontal cortex contribute in systematic yet different ways to human memory processing.

Functional differentiation of medial temporal and frontal regions involved in processing novel and familiar words: an fMRI study

Results of recent functional magnetic resonance imaging (fMRI) studies of memory are not entirely consistent with lesion studies. Furthermore, although imaging probes have identified neural systems associated with processing novel visual episodic information, auditory verbal memory using a novel/familiar paradigm has not yet been examined. To address this gap, fMRI was used to compare the haemodynamic response when listening to recently learned and novel words. Sixteen healthy adults (6 male, 10 female) learned a 10-item word list to 100% criterion, approximately 1 h before functional scanning. During echo-planar imaging, subjects passively listened to a string of words presented at 6-s intervals. Previously learned words were interspersed pseudo-randomly between novel words. Mean scans corresponding to each word type were analysed with a random-effects model using statistical parametric mapping (SPM96). Familiar (learned) words activated the right prefrontal cortex, posterior left parahippocampal gyrus, left medial parietal cortex and right superior temporal gyrus. Novel words activated the anterior left hippocampal region. The results for the familiar words were similar to those found in other functional imaging studies of recognition and retrieval and implicate the right dorsolateral prefrontal and left posterior medial temporal lobe (MTL) regions. The results for novel words require replication, but are consistent with the substantial lesion and PET literature implicating the anterior MTL as a critical site for processing novel episodic information, presumably to permit encoding. Together, these results provide evidence for an anterior-posterior functional differentiation within the MTL in processing novel and familiar verbal information. The differentiation of MTL functions that was obtained is consistent with a large body of PET activation studies but is unique among fMRI studies, which to date have differed from results with PET. Further, the finding of left MTL lateralization is consistent with lesion-based material-specific models of memory.

Direct comparison of episodic encoding and retrieval of words: an event-related fMRI study

Functional magnetic resonance imaging (fMRI) was used to compare directly episodic encoding and retrieval. During encoding, subjects studied visually presented words and reported via keypress whether each word represented a pleasant or unpleasant concept (intentional, deep encoding). During the retrieval phase, subjects indicated (via keypress) whether visually presented words had previously been studied. No reliable differences were found during the recognition phase for words that had been previously studied and those that had not been studied. Areas preferentially active during encoding (relative to retrieval) included left superior frontal cortex, medial frontal cortex, left superior temporal cortex, posterior cingulate, left parahippocampal gyrus, and left inferior frontal gyrus. Regions more active in retrieval than encoding included bilateral inferior parietal cortex, bilateral precuneus, right frontal polar cortex, right dorsolateral prefrontal cortex, and right inferior frontal/insular cortex.

Imaging episodic memory: implications for cognitive theories and phenomena

Functional neuroimaging studies are beginning to identify neuroanatomical correlates of various cognitive functions. This paper presents results relevant to several theories and phenomena of episodic memory, including component processes of episodic retrieval, encoding specificity, inhibition, item versus source memory, encoding-retrieval overlap, and the picture-superiority effect. Overall, by revealing specific activation patterns, the results provide support for existing theoretical views and they add some unique information which may be important to consider in future attempts to develop cognitive theories of episodic memory.

Statistical limitations in functional neuroimaging. I. Non-inferential methods and statistical models

Functional neuroimaging (FNI) provides experimental access to the intact living brain making it possible to study higher cognitive functions in humans. In this review and in a companion paper in this issue, we discuss some common methods used to analyse FNI data. The emphasis in both papers is on assumptions and limitations of the methods reviewed. There are several methods available to analyse FNI data indicating that none is optimal for all purposes. In order to make optimal use of the methods available it is important to know the limits of applicability. For the interpretation of FNI results it is also important to take into account the assumptions, approximations and inherent limitations of the methods used. This paper gives a brief overview over some non-inferential descriptive methods and common statistical models used in FNI. Issues relating to the complex problem of model selection are discussed. In general, proper model selection is a necessary prerequisite for the validity of the subsequent statistical inference. The non-inferential section describes methods that, combined with inspection of parameter estimates and other simple measures, can aid in the process of model selection and verification of assumptions. The section on statistical models covers approaches to global normalization and some aspects of univariate, multivariate, and Bayesian models. Finally, approaches to functional connectivity and effective connectivity are discussed. In the companion paper we review issues related to signal detection and statistical inference.

Right prefrontal cortex and episodic memory retrieval: a functional MRI test of the monitoring hypothesis

Though the right prefrontal cortex is often activated in neuroimaging studies of episodic memory retrieval, the functional significance of this activation remains unresolved. In this functional MRI study of 12 healthy volunteers, we tested the hypothesis that one role of the right prefrontal cortex is to monitor the information retrieved from episodic memory in order to make an appropriate response. The critical comparison was between two word recognition tasks that differed only in whether correct responses did or did not require reference to the spatiotemporal context of words presented during a previous study episode. Activation in a dorsal midlateral region of the right prefrontal cortex was associated with increased contextual monitoring demands, whereas a more ventral region of the right prefrontal cortex showed retrieval-related activation that was independent of task instructions. This functional dissociation of dorsal and ventral right prefrontal regions is discussed in relation to a theoretical framework for the control of episodic memory retrieval.

Recollection and familiarity in recognition memory: an event-related functional magnetic resonance imaging study

The question of whether recognition memory judgments with and without recollection reflect dissociable patterns of brain activity is unresolved. We used event-related, functional magnetic resonance imaging (fMRI) of 12 healthy volunteers to measure hemodynamic responses associated with both studying and recognizing words. Volunteers made one of three judgments to each word during recognition: whether they recollected seeing it during study (R judgments), whether they experienced a feeling of familiarity in the absence of recollection (K judgments), or whether they did not remember seeing it during study (N judgments). Both R and K judgments for studied words were associated with enhanced responses in left prefrontal and left parietal cortices relative to N judgments for unstudied words. The opposite pattern was observed in bilateral temporoccipital regions and amygdalae. R judgments for studied words were associated with enhanced responses in anterior left prefrontal, left parietal, and posterior cingulate regions relative to K judgments. At study, a posterior left prefrontal region exhibited an enhanced response to words subsequently given R versus K judgments, but the response of this region during recognition did not differentiate R and K judgments. K judgments for studied words were associated with enhanced responses in right lateral and medial prefrontal cortex relative to both R judgments for studied words and N judgments for unstudied words, a difference we attribute to greater monitoring demands when memory judgments are less certain. These results suggest that the responses of different brain regions do dissociate according to the phenomenology associated with memory retrieval.

Frontal brain potentials during recognition are modulated by requirements to retrieve perceptual detail

To assess the role of prefrontal cortex in retrieval and address the controversy about whether prefrontal retrieval operations are engaged only following successful retrieval, we recorded event-related brain potentials during two recognition tests with differing demands on retrieval effort. Both tests included object drawings that were (1) identical to those studied, (2) the same but with altered aspect ratios, and (3) previously unseen. Instructions were to respond "old" only if drawings were not modified (specific test) or regardless of modifications (general test). Frontal potentials were enhanced during the specific relative to the general test for all three types of drawings. We conclude that these potentials reflected differential engagement of strategic retrieval, that this function relied on left prefrontal cortex, and that it was not contingent on successful retrieval.

Memory, consciousness and neuroimaging

Neuroimaging techniques that allow the assessment of memory performance in healthy human volunteers while simultaneously obtaining measurements of brain activity in vivo may offer new information on the neural correlates of particular forms of memory retrieval and their association with consciousness and intention. We consider evidence from studies with positron emission tomography and functional magnetic resonance imaging indicating that priming, a form of implicit retrieval, is associated with decreased activity in various cortical regions. We also consider evidence concerning the question of whether two components of explicit retrieval--intentional or effortful search and successful conscious recollection--are preferentially associated with increased activity in prefrontal and medial temporal regions, respectively. Last, we consider recent efforts to probe the relation between the phenomenological character of remembering and neural activity. In this instance we broaden our scope to include studies employing event-related potentials and consider evidence concerning the neural correlates of qualitatively different forms of memory, including memory that is specifically associated with a sense of self, and the recollection of particular temporal or perceptual features that might contribute to a rich and vivid experience of the past.

Neural correlates of memory retrieval during recognition memory and cued recall

Regional brain activity, measured by H215O PET, was investigated during recognition memory and word-stem cued recall of words in order to compare the neural correlates of two components of memory retrieval-effort and success-as a function of task. For each task there was a baseline and two retrieval conditions. In one retrieval condition (zero density), none of the test items corresponded to words encoded in a preceding study phase. Differences in activity between this condition and the baseline were employed to characterize the neural correlates of retrieval effort in each task. In the other retrieval condition (high density), 80% of the test items had been studied previously. Differences in brain activity between this condition and the zero-density condition were taken to represent the neural correlates of successful retrieval. The principal findings concern the right anterior prefrontal cortex, a region demonstrated previously to be active during episodic retrieval. Relative to baseline, this region showed no evidence of activation in the zero-density condition of the recognition task, but did show enhanced activity in the equivalent condition of the cued-recall task. In contrast, relative to the zero-density condition, the high-density condition was associated with increased right prefrontal activity during recognition, but reduced activity during cued recall. It is proposed that the right prefrontal cortex supports cognitive processes that operate on information retrieved in response to a test item and that these processes contribute to the evaluation of whether the information represents an appropriate prior episode.

Building memories: remembering and forgetting of verbal experiences as predicted by brain activity

A fundamental question about human memory is why some experiences are remembered whereas others are forgotten. Brain activation during word encoding was measured using blocked and event-related functional magnetic resonance imaging to examine how neural activation differs for subsequently remembered and subsequently forgotten experiences. Results revealed that the ability to later remember a verbal experience is predicted by the magnitude of activation in left prefrontal and temporal cortices during that experience. These findings provide direct evidence that left prefrontal and temporal regions jointly promote memory formation for verbalizable events.