Conceptual and visual features contribute to visual memory for natural images

Ex-Gaussian vector metric and similarity index applied to reaction time analysis

In psychology and cognitive neuroscience, reaction time (RT) series and their ex-Gaussian distributions are commonly used as scalar quantities to explore the time course of attentional processes. However, we propose that such attentional processes can also be analyzed using an "ex-Gaussian vector", defined by successive triads of ex-Gaussian sigma, tau, and mu parameters from RT series. This geometrical object may help characterize interindividual differences between congruent and incongruent stimuli in the attentional Stroop task within a group of participants. To test these hypotheses, we calculated the similarity index of these geometrical objects in young adults without detectable neurological disorders. Our findings show that during two weeks of continuous Stroop task application, each participant displayed distinct ex-Gaussian RT vectors in a Cartesian 3D plot. Furthermore, our study found that the similarity index between ex-Gaussian RT vectors was significantly higher for incongruent stimuli than for congruent stimuli.

Low detail retention in visual memory despite focused effort

Humans can recognize a vast number of previously seen images, yet their ability to recall fine details from visual memory remains limited. This study investigated whether prolonged study of a small number of stimuli could improve the recognition accuracy for memorizing details of the scene. We developed a novel experimental paradigm that allowed repeated testing of memory for individual images, allowing us to query images repeatedly and measure which parts of the scene were remembered, and which were forgotten. Our results revealed that participants struggled to achieve high accuracy in detail-oriented memory tasks, even with extensive effort and focus. Follow-up experiments explored potential factors contributing to this limitation, shedding light on why memorizing fine details is inherently difficult. These findings underscore the challenges of achieving high-detail visual memory in long-term memory for complex scenes-although we can memorize large numbers of scenes with low fidelity, we cannot memorize details even in a small number of scenes.

Predicting image memorability from evoked feelings

While viewing a visual stimulus, we often cannot tell whether it is inherently memorable or forgettable. However, the memorability of a stimulus can be quantified and partially predicted by a collection of conceptual and perceptual factors. Higher-level properties that represent the "meaningfulness" of a visual stimulus to viewers best predict whether it will be remembered or forgotten across a population. Here, we hypothesize that the feelings evoked by an image, operationalized as the valence and arousal dimensions of affect, significantly contribute to the memorability of scene images. We ran two complementary experiments to investigate the influence of affect on scene memorability, in the process creating a new image set (VAMOS) of hundreds of natural scene images for which we obtained valence, arousal, and memorability scores. From our first experiment, we found memorability to be highly reliable for scene images that span a wide range of evoked arousal and valence. From our second experiment, we found that both valence and arousal are significant but weak predictors of image memorability. Scene images were most memorable if they were slightly negatively valenced and highly arousing. Images that were extremely positive or unarousing were most forgettable. Valence and arousal together accounted for less than 8% of the variance in image memorability. These findings suggest that evoked affect contributes to the overall memorability of a scene image but, like other singular predictors, does not fully explain it. Instead, memorability is best explained by an assemblage of visual features that combine, in perhaps unintuitive ways, to predict what is likely to stick in our memory.

Convolutional neural networks uncover the dynamics of human visual memory representations over time

The ability to accurately retrieve visual details of past events is a fundamental cognitive function relevant for daily life. While a visual stimulus contains an abundance of information, only some of it is later encoded into long-term memory representations. However, an ongoing challenge has been to isolate memory representations that integrate various visual features and uncover their dynamics over time. To address this question, we leveraged a novel combination of empirical and computational frameworks based on the hierarchal structure of convolutional neural networks and their correspondence to human visual processing. This enabled to reveal the contribution of different levels of visual representations to memory strength and their dynamics over time. Visual memory strength was measured with distractors selected based on their shared similarity to the target memory along low or high layers of the convolutional neural network hierarchy. The results show that visual working memory relies similarly on low and high-level visual representations. However, already after a few minutes and on to the next day, visual memory relies more strongly on high-level visual representations. These findings suggest that visual representations transform from a distributed to a stronger high-level conceptual representation, providing novel insights into the dynamics of visual memory over time.

The upside of cumulative conceptual interference on exemplar-level mnemonic discrimination

Although long-term visual memory (LTVM) has a remarkable capacity, the fidelity of its episodic representations can be influenced by at least two intertwined interference mechanisms during the encoding of objects belonging to the same category: the capacity to hold similar episodic traces (e.g., different birds) and the conceptual similarity of the encoded traces (e.g., a sparrow shares more features with a robin than with a penguin). The precision of episodic traces can be tested by having participants discriminate lures (unseen objects) from targets (seen objects) representing different exemplars of the same concept (e.g., two visually similar penguins), which generates interference at retrieval that can be solved if efficient pattern separation happened during encoding. The present study examines the impact of within-category encoding interference on the fidelity of mnemonic object representations, by manipulating an index of cumulative conceptual interference that represents the concurrent impact of capacity and similarity. The precision of mnemonic discrimination was further assessed by measuring the impact of visual similarity between targets and lures in a recognition task. Our results show a significant decrement in the correct identification of targets for increasing interference. Correct rejections of lures were also negatively impacted by cumulative interference as well as by the visual similarity with the target. Most interestingly though, mnemonic discrimination for targets presented with a visually similar lure was more difficult when objects were encoded under lower, not higher, interference. These findings counter a simply additive impact of interference on the fidelity of object representations providing a finer-grained, multi-factorial, understanding of interference in LTVM.

Differential effects of intrinsic properties of natural scenes and interference mechanisms on recognition processes in long-term visual memory

Humans display remarkable long-term visual memory (LTVM) processes. Even though images may be intrinsically memorable, the fidelity of their visual representations, and consequently the likelihood of successfully retrieving them, hinges on their similarity when concurrently held in LTVM. In this debate, it is still unclear whether intrinsic features of images (perceptual and semantic) may be mediated by mechanisms of interference generated at encoding, or during retrieval, and how these factors impinge on recognition processes. In the current study, participants (32) studied a stream of 120 natural scenes from 8 semantic categories, which varied in frequencies (4, 8, 16 or 32 exemplars per category) to generate different levels of category interference, in preparation for a recognition test. Then they were asked to indicate which of two images, presented side by side (i.e. two-alternative forced-choice), they remembered. The two images belonged to the same semantic category but varied in their perceptual similarity (similar or dissimilar). Participants also expressed their confidence (sure/not sure) about their recognition response, enabling us to tap into their metacognitive efficacy (meta-d'). Additionally, we extracted the activation of perceptual and semantic features in images (i.e. their informational richness) through deep neural network modelling and examined their impact on recognition processes. Corroborating previous literature, we found that category interference and perceptual similarity negatively impact recognition processes, as well as response times and metacognitive efficacy. Moreover, images semantically rich were less likely remembered, an effect that trumped a positive memorability boost coming from perceptual information. Critically, we did not observe any significant interaction between intrinsic features of images and interference generated either at encoding or during retrieval. All in all, our study calls for a more integrative understanding of the representational dynamics during encoding and recognition enabling us to form, maintain and access visual information.

The effect of encoding task on the forgetting of object gist and details

One important feature of episodic memory is that it contains fine-grained and vividly recollected details. How to improve and maintain detailed information over time has been one of the central issues in memory research. Previous studies have inconsistent findings on whether detailed memory is forgotten more rapidly than gist memory. In this study, we investigated to what extent different encoding tasks modulated forgetting of gist and detailed information. In three experiments, participants were presented pictures of common objects and were asked to name them (Experiment 1), describe the details about them (Experiment 2) or imagine scenes associated with them (Experiment 3). After intervals of 10 minutes, one day, one week and one month, gist and detailed memories of the pictures were tested and assessed using a remember/know/guess judgement. The results showed that after the naming task, gist and detailed memories were forgotten at a similar rate, but after the description and the imagination tasks, detailed memory was forgotten at a slower rate than gist memory. The forgetting rate of gist memory was the slowest after the naming task, while that of detailed memory was the slowest after the description task. In addition, when three experiments were compared, the naming task enhanced the contributions of recollection and familiarity for gist memory, while the description task enhanced the contribution of familiarity for detailed memory. These results reveal the importance of the encoding task in the forgetting of gist and detailed information, and suggest a possible way to maintain perceptual details of objects at longer intervals.

Multiple Exposures Enhance Both Item Memory and Contextual Memory Over Time

Repetition learning is an efficient way to enhance memory performance in our daily lives and educational practice. However, it is unclear to what extent repetition or multiple exposures modulate different types of memory over time. The inconsistent findings on it may be associated with encoding strategy. In this study, participants were presented with pairs of pictures (same, similar, and different) once (see section “Experiment 1”) or three times (see section “Experiment 2”) and were asked to make a same/similar/different judgment. By this, an elaborative encoding is more required for the “same” and “similar” conditions than the “different” condition. Then after intervals of 10 min, 1 day, and 1 week, they were asked to perform a recognition test to discriminate a repeated and a similar picture, followed by a remember/know/guess assessment and a contextual judgment. The results showed that after learning the objects three times, both item memory and contextual memory improved. Multiple exposures enhanced the hit rate for the “same” and “similar” conditions, but did not change the false alarm rate significantly. The recollection, rather than the familiarity, contributed to the repetition effect. In addition, the memory enhancement was manifested in each encoding condition and retention interval, especially for the “same” condition and at 10-min and 1-day intervals. These results clarify how repetition influences item and contextual memories during discriminative learning and suggest that multiple exposures render the details more vividly remembered and retained over time when elaborative encoding is emphasized.

Image memorability is predicted by discriminability and similarity in different stages of a convolutional neural network

The features of an image can be represented at multiple levels-from its low-level visual properties to high-level meaning. What drives some images to be memorable while others are forgettable? We address this question across two behavioral experiments. In the first, different layers of a convolutional neural network (CNN), which represent progressively higher levels of features, were used to select the images that would be shown to 100 participants through a form of prospective assignment. Here, the discriminability/similarity of an image with others, according to different CNN layers dictated the images presented to different groups, who made a simple indoor versus outdoor judgment for each scene. We found that participants remember more scene images that were selected based on their low-level discriminability or high-level similarity. A second experiment replicated these results in an independent sample of 50 participants, with a different order of postencoding tasks. Together, these experiments provide evidence that both discriminability and similarity, at different visual levels, predict image memorability.

False memories for scenes using the DRM paradigm

People are remarkably good at remembering photographs. To further investigate the nature of the stored representations and the fidelity of human memories, it would be useful to evaluate the visual similarity of stimuli presented in experiments. Here, we explored the possible use of convolutional neural networks (CNN) as a measure of perceptual or representational similarity of visual scenes with respect to visual memory research. In Experiment 1, we presented participants with sets of nine images from the same scene category and tested whether they were able to detect the most distant scene in the image space defined by CNN. Experiment 2 was a visual variant of the Deese-Roediger-McDermott paradigm. We asked participants to remember a set of photographs from the same scene category. The photographs were preselected based on their distance to a particular visual prototype (defined as centroid of the image space). In the recognition test, we observed higher false alarm rates for scenes closer to this visual prototype. Our findings show that the similarity measured by CNN is reflected in human behavior: people can detect odd-one-out scenes or be lured to false alarms with similar stimuli. This method can be used for further studies regarding visual memory for complex scenes.

Discriminative learning of similar objects enhances memory for the objects and contexts

How to improve our episodic memory is an important issue in the field of memory. In the present study, we used a discriminative learning paradigm that was similar to a paradigm used in animal studies. In Experiment 1, a picture (e.g., a dog) was either paired with an identical picture, with a similar picture of the same concept (e.g., another dog), or with a picture of a different concept (e.g., a cat). Then, after intervals of 10 min, 1 d, and 1 wk, participants were asked to perform a 2-alternative forced-choice (2AFC) task to discriminate between a repeated and a similar picture, followed by the contextual judgment. In Experiment 2, eye movements were measured when participants encoded the pairs of pictures. The results showed that by discriminative learning, there was better memory performance in the 2AFC task for the “same” and “similar” conditions than for the “different” condition. In addition, there was better contextual memory performance for the “similar” condition than for the other two conditions. With regard to the eye movements, the participants were more likely to fixate on the lure objects and made more saccades between the target and lure objects in the “similar” (versus “different”) condition. The number of saccades predicted how well the targets were remembered in both the 2AFC and contextual memory tasks. These results suggested that with discriminative learning of similar objects, detailed information could be better encoded by distinguishing the object from similar interferences, making the details and the contexts better remembered and retained over time.

Briefly Flashed Scenes Can Be Stored in Long-Term Memory

The capacity of human memory is impressive. Previous reports have shown that when asked to memorize images, participants can recognize several thousands of visual objects in great details even with a single viewing of a few seconds per image. In this experiment, we tested recognition performance for natural scenes that participants saw for 20 ms only once (untrained group) or 22 times over many days (trained group) in an unrelated task. 400 images (200 previously viewed and 200 novel images) were flashed one at a time and participants were asked to lift their finger from a pad whenever they thought they had already seen the image (go/no-go paradigm). Compared to previous reports of excellent recognition performance with only single presentations of a few seconds, untrained participants were able to recognize only 64% of the 200 images they had seen few minutes before. On the other hand, trained participants, who had processed the flashed images (20 ms) several times, could correctly recognize 89% of them. EEG recordings confirmed these behavioral results. As early as 230 ms after stimulus onset, a significant event-related-potential (ERP) difference between familiar and new images was observed for the trained but not for the untrained group. These results show that briefly flashed unmasked scenes can be incidentally stored in long-term memory when repeated.

Format change and semantic relatedness effects on the ERP correlates of recognition: old pairs, new pairs, different stories

In this event-related potential (ERP) study, we investigated the effects of format change and semantic relatedness in a recognition task using pairs composed of a word and a line drawing. The semantic relatedness of the pairs (related: rabbit-carrot; unrelated: duck-artichoke) influenced their associative properties and corresponding distinctiveness, while format change refers to the switching of an item from the verbal form to the line drawing form between study and recognition (e.g., the word "egg" is associated with a drawing of a hen at study, and a line drawing of an egg is associated with the word "hen" at test). Study-test format change thus prevents visual matching while maintaining conceptual matching. While the N300 potential was only modulated by the semantic relatedness of the pair, both factors modulated recognition performance and corresponding ERP old/new effects with larger mid-frontal N400 old/new effect (300-500 ms) and larger parietal old/new effect (500-800 ms) in the same compared to the different-format condition, as well as for related compared to unrelated pairs. Furthermore, the semantic relatedness of correctly recognized old pairs modulated the anterior N400 while it modulated the posterior N400 for correctly rejected pairs. These results suggest that semantic relatedness and familiarity related to the amount of change between study and test present distinct ERP signatures in the N400 window. They suggest also that the distinctiveness and the ease of the retrieval of the pair could be determining for the parietal old/new effect.

Pre-Feedback Risk Expectancies and Reception of Low-Risk Health Feedback: Absolute and Comparative Lack of Reassurance

BACKGROUND

Personalised health-risk assessment is one of the most common components of health promotion programs. Previous research on responses to health risk feedback has commonly focused on the reception of bad news (high-risk feedback). The reception of low-risk feedback has been comparably neglected since it is assumed that good news is reassuring and readily received. However, field studies suggest mixed responses to low-risk health feedback. Accordingly, we examine whether pre-feedback risk expectancies can mitigate the reassuring effects of good news.

METHODS

In two studies (N = 187, N = 565), after assessing pre-feedback risk expectancies, participants received low-risk personalised feedback about their own risk of developing (the fictitious) Tucson Chronic Fatigue Syndrome (TCFS). Study 2 also included peer TCFS risk status feedback. Afterwards, self- and peer-related risk perception for TCFS was assessed.

RESULTS

In both studies, participants who expected to be at high risk but received good news (unexpected low-risk feedback) showed absolute lack of reassurance. Specifically, they felt at significantly greater TCFS risk than participants who received expected good news. Moreover, the unexpected low-risk group even believed that their risk was as high as (Study 1) or higher (Study 2) than that of their peers (comparative lack of reassurance).

CONCLUSION

Results support the notion that high pre-feedback risk expectancies can mitigate absolute and comparative reassuring effects of good news.

Being and feeling liked by others: how social inclusion impacts health

This study examined the effects of perceived and actual social inclusion on health across and within individuals from a network perspective. During the first semester, 75 freshmen students provided bi-weekly ratings on their perceived social inclusion and health. To capture actual social inclusion, each student nominated liked and disliked fellow students. Perceived social inclusion mediated the effect of actual social inclusion on health. Specifically, students with more 'likes' perceived more social inclusion and those with higher perceived inclusion reported a better health status (between-person effect). In addition, at time points, when students received more 'likes' they also perceived more social inclusion. They reported better health at times when they felt more included (within-person effect). Thus, the perception of social inclusion is rooted in reality and actual social inclusion has an impact on health when passing the filter of perception.

Impaired picture recognition in transient epileptic amnesia

Transient epileptic amnesia (TEA) is an epileptic syndrome characterized by recurrent, brief episodes of amnesia. Transient epileptic amnesia is often associated with the rapid decline in recall of new information over hours to days (accelerated long-term forgetting - 'ALF'). It remains unknown how recognition memory is affected in TEA over time. Here, we report a systematic study of picture recognition in patients with TEA over the course of one week. Sixteen patients with TEA and 16 matched controls were presented with 300 photos of everyday life scenes. Yes/no picture recognition was tested 5min, 2.5h, 7.5h, 24h, and 1week after picture presentation using a subset of target pictures as well as similar and different foils. Picture recognition was impaired in the patient group at all test times, including the 5-minute test, but it declined normally over the course of 1week. This impairment was associated predominantly with an increased false alarm rate, especially for similar foils. High performance on a control test indicates that this impairment was not associated with perceptual or discrimination deficits. Our findings suggest that, at least in some TEA patients with ALF in verbal recall, picture recognition does not decline more rapidly than in controls over 1week. However, our findings of an early picture recognition deficit suggest that new visual memories are impoverished after minutes in TEA. This could be the result of deficient encoding or impaired early consolidation. The early picture recognition deficit observed could reflect either the early stages of the process that leads to ALF or a separable deficit of anterograde memory in TEA. Lastly, our study suggests that at least some patients with TEA are prone to falsely recognizing new everyday visual information that they have not in fact seen previously. This deficit, alongside their ALF in free recall, likely affects everyday memory performance.

Visual recognition memory test performance was improved in older adults by extending encoding time and repeating test trials

OBJECTIVES

The aim of this study was to investigate whether the combination of extension of the encoding time and repetition of a test trial would improve the visual recognition memory performance in older adults.

METHODS

We evaluated visual memory performance in young and older adults on a Yes-No recognition memory test under four different conditions. The conditions consisted of combinations of encoding times of two and four seconds (E2 and E4) and first and second retrieval practice test trials (T1 and T2): E2T1, E2T2, E4T1 and E4T2. Performance was evaluated by measuring hit rates, false alarm rates, discrimination ability and response bias.

RESULTS

Older adults showed better improvement of hit rate and discrimination ability under the E4T2 conditions whereas young adults showed better memory performance under the E2T2 conditions.

CONCLUSIONS

A longer encoding time and repetition of the test was effective in improving the visual memory performance in terms of the hit rates and discrimination ability of older adults. The results suggest that this strategy should be useful in providing a suitable work environment for older workers.

A visual object stimulus database with standardized similarity information

Although many visual stimulus databases exist, none has data on item similarity levels for multiple items of each kind of stimulus. We present such data for 50 sets of grayscale object photographs. Similarity measures between pictures in each set (e.g., 25 different buttons) were collected using a similarity-sorting method (Goldstone, Behavior Research Methods Instruments & Computers, 26(4):381-386, 1994). A validation experiment used data from 1 picture set and compared responses from standard pairwise measures. This showed close agreement. The similarity-sorting measures were then standardized across picture sets, using pairwise ratings. Finally, the standardized similarity distances were validated in a recognition memory experiment; false alarms increased when targets and foils were more similar. These data will facilitate memory and perception research that needs to make comparisons between stimuli with a range of known target-foil similarities.

Social curiosity and gossip: related but different drives of social functioning

The present online-questionnaire study examined two fundamental social behaviors, social curiosity and gossip, and their interrelations in an English (n = 218) and a German sample (n = 152). Analyses showed that both samples believed that they are less gossipy but more curious than their peers. Multidimensional SEM of self and trait conceptions indicated that social curiosity and gossip are related constructs but with different patterns of social functions. Gossip appears to serve predominantly entertainment purposes whereas social curiosity appears to be more driven by a general interest in gathering information about how other people feel, think, and behave and the need to belong. Relationships to other personality traits (N, E, O) provided additional evidence for divergent validity. The needs for gathering and disseminating social information might represent two interlinked but different drives of cultural learning.

On the contribution of binocular disparity to the long-term memory for natural scenes

Binocular disparity is a fundamental dimension defining the input we receive from the visual world, along with luminance and chromaticity. In a memory task involving images of natural scenes we investigate whether binocular disparity enhances long-term visual memory. We found that forest images studied in the presence of disparity for relatively long times (7s) were remembered better as compared to 2D presentation. This enhancement was not evident for other categories of pictures, such as images containing cars and houses, which are mostly identified by the presence of distinctive artifacts rather than by their spatial layout. Evidence from a further experiment indicates that observers do not retain a trace of stereo presentation in long-term memory.