Perceived Synchrony of Frog Multimodal Signal Components Is Influenced by Content and Order

Alteration of the temporal association between courtship audio and visual components affects female sexual response

Some multimodal signals-that is, occurring in more than one sensory modality-appear to carry additional information which is not present when component signals are presented separately. To understand the function of male ring dove's (Streptopelia risoria) multimodal courtship, we used audiovisual playback of male displays to investigate female response to stimuli differing in their audiovisual timing. From natural courtship recordings, we created a shifted stimulus where audio was shifted relative to video by a fixed value and a jittered stimulus, where each call was moved randomly along the visual channel. We presented 3 groups of females with the same stimulus type, that is, control, shifted, and jittered, for 7 days. We recorded their behavior and assessed pre- and post-test blood estradiol concentration. We found that playback exposure increased estradiol levels, confirming that this technique can be efficiently used to study doves' sexual communication. Additionally, chasing behavior (indicating sexual stimulation) increased over experimental days only in the control condition, suggesting a role of multimodal timing on female response. This stresses the importance of signal configuration in multimodal communication, as additional information is likely to be contained in the temporal association between modalities.

Can you hear/see me? Multisensory integration of signals does not always facilitate mate choice

Females of many species choose mates using multiple sensory modalities. Multimodal noise may arise, however, in dense aggregations of animals communicating via multiple sensory modalities. Some evidence suggests multimodal signals may not always improve receiver decision-making performance. When sensory systems process input from multimodal signal sources, multimodal noise may arise and potentially complicate decision-making due to the demands on cognitive integration tasks. We tested female túngara frog, Physalaemus (=Engystomops) pustulosus, responses to male mating signals in noise from multiple sensory modalities (acoustic and visual). Noise treatments were partitioned into three categories: acoustic, visual, and multimodal. We used natural calls from conspecifics and heterospecifics for acoustic noise. Robotic frogs were employed as either visual signal components (synchronous vocal sac inflation with call) or visual noise (asynchronous vocal sac inflation with call). Females expressed a preference for the typically more attractive call in the presence of unimodal noise. However, during multimodal signal and noise treatments (robofrogs employed with background noise), females failed to express a preference for the typically attractive call in the presence of conspecific chorus noise. We found that social context and temporal synchrony of multimodal signaling components are important for multimodal communication. Our results demonstrate that multimodal signals have the potential to increase the complexity of the sensory scene and reduce the efficacy of female decision making.

Cross-modal facilitation of auditory discrimination in a frog

Stimulation in one sensory modality can affect perception in a separate modality, resulting in diverse effects including illusions in humans. This can also result in cross-modal facilitation, a process where sensory performance in one modality is improved by stimulation in another modality. For instance, a simple sound can improve performance in a visual task in both humans and cats. However, the range of contexts and underlying mechanisms that evoke such facilitation effects remain poorly understood. Here, we demonstrated cross-modal stimulation in wild-caught túngara frogs, a species with well-studied acoustic preferences in females. We first identified that a combined visual and seismic cue (vocal sac movement and water ripple) was behaviourally relevant for females choosing between two courtship calls in a phonotaxis assay. We then found that this combined cross-modal stimulus rescued a species-typical acoustic preference in the presence of background noise that otherwise abolished the preference. These results highlight how cross-modal stimulation can prime attention in receivers to improve performance during decision-making. With this, we provide the foundation for future work uncovering the processes and conditions that promote cross-modal facilitation effects.

Open-source five degree of freedom motion platform for investigating fish-robot interaction

This paper presents the design, construction, operation, and validation of a robotic gantry platform specifically designed for studying fish-robot interaction. The platform has five degrees of freedom to manipulate the three-dimensional position, yaw angle, and the pitch of a lure. Additionally, it has a four-conductor slip ring that allows power and data to be transmitted to the lure for the operation of fins and other actuators that increase realism or act as stimuli to focal fish during an ethorobotic experiment. The design is open-source, low-cost, and includes purpose-built electronics, software, and hardware to make it extensible and customizable for a number of applications with varying requirements.

Evolution and function of multimodal courtship displays

Courtship displays are behaviours aimed to facilitate attraction and mating with the opposite sex and are very common across the animal kingdom. Most courtship displays are multimodal, meaning that they are composed of concomitant signals occurring in different sensory modalities. Although courtship often strongly influences reproductive success, the question of why and how males use multimodal courtship to increase their fitness has not yet received much attention. Very little is known about the role of different components of male courtship and their relative importance for females. Indeed, most of the work on courtship displays have focused on effects on female choice, often neglecting other possible roles. Additionally, a number of scientists have recently stressed the importance of considering the complexity of a display and the interactions between its different components in order to grasp all the information contained in those multimodal signals. Unfortunately, these methods have not yet been extensively adapted in courtship studies. The aim of this study was to review what is currently known about the functional significance of courtship displays, particularly about the role of multimodality in the courtship communication context. Emphasis is placed on those cases where a complete picture of the communication system can only be assessed by taking complexity and interaction between different modalities into account.

Synchronization of speed, sound and iridescent color in a hummingbird aerial courtship dive

Many animal signals are complex, often combining multimodal components with dynamic motion. To understand the function and evolution of these displays, it is vital to appreciate their spatiotemporal organization. Male broad-tailed hummingbirds (Selasphorus platycercus) perform dramatic U-shaped courtship dives over females, appearing to combine rapid movement and dive-specific mechanical noises with visual signals from their iridescent gorgets. To understand how motion, sound and color interact in these spectacular displays, we obtained video and audio recordings of dives performed by wild hummingbirds. We then applied a multi-angle imaging technique to estimate how a female would perceive the male's iridescent gorget throughout the dive. We show that the key physical, acoustic and visual aspects of the dive are remarkably synchronized-all occurring within 300 milliseconds. Our results highlight the critical importance of accounting for motion and orientation when investigating animal displays: speed and trajectory affect how multisensory signals are produced and perceived.