Decoding humor experiences from brain activity of people viewing comedy movies

Integrating media content analysis, reception analysis, and media effects studies

Every day, the world of media is at our fingertips, whether it is watching movies, listening to the radio, or browsing online media. On average, people spend over 8 h per day consuming messages from the mass media, amounting to a total lifetime dose of more than 20 years in which conceptual content stimulates our brains. Effects from this flood of information range from short-term attention bursts (e.g., by breaking news features or viral 'memes') to life-long memories (e.g., of one's favorite childhood movie), and from micro-level impacts on an individual's memory, attitudes, and behaviors to macro-level effects on nations or generations. The modern study of media's influence on society dates back to the 1940s. This body of mass communication scholarship has largely asked, "what is media's effect on the individual?" Around the time of the cognitive revolution, media psychologists began to ask, "what cognitive processes are involved in media processing?" More recently, neuroimaging researchers started using real-life media as stimuli to examine perception and cognition under more natural conditions. Such research asks: "what can media tell us about brain function?" With some exceptions, these bodies of scholarship often talk past each other. An integration offers new insights into the neurocognitive mechanisms through which media affect single individuals and entire audiences. However, this endeavor faces the same challenges as all interdisciplinary approaches: Researchers with different backgrounds have different levels of expertise, goals, and foci. For instance, neuroimaging researchers label media stimuli as "naturalistic" although they are in many ways rather artificial. Similarly, media experts are typically unfamiliar with the brain. Neither media creators nor neuroscientifically oriented researchers approach media effects from a social scientific perspective, which is the domain of yet another species. In this article, we provide an overview of approaches and traditions to studying media, and we review the emerging literature that aims to connect these streams. We introduce an organizing scheme that connects the causal paths from media content → brain responses → media effects and discuss network control theory as a promising framework to integrate media content, reception, and effects analyses.

Naturalistic Stimuli in Affective Neuroimaging: A Review

Naturalistic stimuli such as movies, music, and spoken and written stories elicit strong emotions and allow brain imaging of emotions in close-to-real-life conditions. Emotions are multi-component phenomena: relevant stimuli lead to automatic changes in multiple functional components including perception, physiology, behavior, and conscious experiences. Brain activity during naturalistic stimuli reflects all these changes, suggesting that parsing emotion-related processing during such complex stimulation is not a straightforward task. Here, I review affective neuroimaging studies that have employed naturalistic stimuli to study emotional processing, focusing especially on experienced emotions. I argue that to investigate emotions with naturalistic stimuli, we need to define and extract emotion features from both the stimulus and the observer.

Movies and narratives as naturalistic stimuli in neuroimaging

Using movies and narratives as naturalistic stimuli in human neuroimaging studies has yielded significant advances in understanding of cognitive and emotional functions. The relevant literature was reviewed, with emphasis on how the use of naturalistic stimuli has helped advance scientific understanding of human memory, attention, language, emotions, and social cognition in ways that would have been difficult otherwise. These advances include discovering a cortical hierarchy of temporal receptive windows, which supports processing of dynamic information that accumulates over several time scales, such as immediate reactions vs. slowly emerging patterns in social interactions. Naturalistic stimuli have also helped elucidate how the hippocampus supports segmentation and memorization of events in day-to-day life and have afforded insights into attentional brain mechanisms underlying our ability to adopt specific perspectives during natural viewing. Further, neuroimaging studies with naturalistic stimuli have revealed the role of the default-mode network in narrative-processing and in social cognition. Finally, by robustly eliciting genuine emotions, these stimuli have helped elucidate the brain basis of both basic and social emotions apparently manifested as highly overlapping yet distinguishable patterns of brain activity.

Task-dependent fMRI decoder with the power to extend Gabor patch results to Natural images

Scientists are often asked to what extent a simple finding in a laboratory can be generalized to complicated phenomena in our daily lives. The same is equally true of vision science; numerous critical discoveries about our visual system have been made using very simple visual images, such as Gabor patches, but to what extent can these findings be applied to more natural images? Here, we used the fMRI decoding technique and directly tested whether the findings obtained with primitive visual stimuli (Gabor patches) were applicable to natural images. In the fMRI experiments, participants performed depth and resolution tasks with both Gabor patches and natural images. We created a fMRI decoder made from the results of the Gabor patch experiments that classified a brain activity pattern into the depth or resolution task, and then examined how successful the task-dependent decoder could sort a brain activity pattern in the natural image experiment into the depth or resolution task. As a result, we found that the task-dependent decoder constructed from Gabor patch experiments could predict which task (depth or resolution task) a participant was engaged in the natural image experiments, especially in the V3 and middle temporal (MT+) areas of the brain. This is consistent with previous researches on the cortical activation relating to depth perception rather than perceptual processing of display resolution. These results provide firm evidence that fMRI decoding technique possesses the power to evaluate the application of Gabor patch results (laboratory findings) to the natural images (everyday affairs), representing a new approach for studying the mechanism of visual perception.

Humor Appreciation Involves Parametric and Synchronized Activity in the Medial Prefrontal Cortex and Hippocampus

Humor perception is a ubiquitous phenomenon in human societies. In theories of humor perception, three factors, non-seriousness, social context, and incongruity, have been implicated in humor. In another theory, however, elaboration and reinterpretation of contexts are considered to play a role in eliciting humor. Although the neural correlates of humor appreciation have been investigated using neuroimaging methods, only a few studies have conducted such experiments under natural conditions. In the present study, two functional magnetic resonance imaging experiments, using a comedy movie as a stimulus, were conducted to investigate the neural correlates of humor under natural conditions. The subjects' brain activity was measured while watching and enjoying a movie. In experiment 1, a parametric analysis showed that the medial prefrontal cortex (MPFC) and hippocampus/amygdala had a positive relationship with the subjective rating of funniness. In experiment 2, intersubject correlation was analyzed to investigate synchronized activity across all participants. Signal synchronization that paralleled increased funniness ratings was observed in the MPFC and hippocampus. Thus, it appears that both parametric and synchronized activity in the MPFC and hippocampus are important during humor appreciation. The present study has revealed the brain regions that are predominantly involved in humor sensation under natural condition.

A studyforrest extension, simultaneous fMRI and eye gaze recordings during prolonged natural stimulation

Here we present an update of the studyforrest (http://studyforrest.org) dataset that complements the previously released functional magnetic resonance imaging (fMRI) data for natural language processing with a new two-hour 3 Tesla fMRI acquisition while 15 of the original participants were shown an audio-visual version of the stimulus motion picture. We demonstrate with two validation analyses that these new data support modeling specific properties of the complex natural stimulus, as well as a substantial within-subject BOLD response congruency in brain areas related to the processing of auditory inputs, speech, and narrative when compared to the existing fMRI data for audio-only stimulation. In addition, we provide participants' eye gaze location as recorded simultaneously with fMRI, and an additional sample of 15 control participants whose eye gaze trajectories for the entire movie were recorded in a lab setting—to enable studies on attentional processes and comparative investigations on the potential impact of the stimulation setting on these processes.

Patients' Evaluation of Intervention by a Medical Clown Visit or by Viewing a Humorous Film Following In Vitro Fertilization and Embryo Transfer

This study compared responses to an in-person clown visit and a humorous film following in vitro fertilization and embryo transfer. Intervention was a 10-minute clown visit (n = 101) or 10-minute humorous video clip (n = 99). Demographic and fertility-related data and preintervention anxiety scores were collected. Participants completed an Evaluation of Intervention form postintervention. There were no group differences on demographic or fertility-related data or anxiety scores. Findings indicate while participants viewed the intervention positively, the clown visit offered a higher degree of satisfaction in more patients than did the film. Median evaluation scores were significantly higher for the clown visit, specifically reducing anxiety level and being more distracting. Both groups reported that the exposure made the clinic experience more pleasant and did not bother them, and most would recommend incorporating the intervention in routine treatment. However, free-text comments clearly expressed greater enthusiasm to the in-person clown intervention than to the film.