Crossmodal comparisons of signal components allow for relative-distance assessment

Modified specific components of conspecific advertisement calls influence behavioral and neural responses in music frogs

Vocal communication plays a critical role in the transfer and exchange of information among animals. However, it remains unclear how modifications to specific call components simultaneously affect behavioral and neural responses. To address these issues, we conducted phonotaxis experiments and neural signal recordings in Emei music frogs (Nidirana daunchina), exposing them to auditory stimuli with varying degrees of information coherence violations. During the electrophysiological recordings, we also presented stimuli with altered physical properties featuring rising intonation. The phonotaxis experiments showed that females exhibited reduced attraction to altered calls with potential information coherence violations, suggesting that information coherence may influence female choice. Similarly, the electrophysiological experiments indicated a correlation between the amplitudes of the N400 and late positive components (LPC) with information incongruity and altered physical properties, respectively. Notably, the N400 amplitudes increased proportionally with the extent of potential information coherence violations. Given that N400 is a well-established neural indicator for prediction error in perceptual processes, including semantic processing in humans, and considering the significant evolutionary conservation of brain structure and function among vertebrates, these findings suggest that information coherence contained in the calls plays a crucial role in anuran vocal communication.

Concordance of movements and songs enhances receiver responses to multimodal display in the starling

Many animals produce signals that consist of vocalizations and movements to attract mates or deter rivals. We usually consider them as components of a single multimodal signal because they are temporally coordinated. Sometimes, however, this relationship takes on a more complex spatiotemporal character, resembling choreographed music. Timing is important for audio-visual integration, but choreographic concordance requires even more skill and competence from the signaller. Concordance should therefore have a strong impact on receivers; however, little is known about its role in audio-visual perception during natural interactions. We studied the effects of movement and song type concordance in audio-visual displays of the starling, Sturnus vulgaris. Starlings produce two types of movements that naturally appear in specific phrases of songs with a similar temporal structure and amplitude. In an experiment with a taxidermic robotic model, males responded more to concordant audio-visual displays, which are also naturally preferred, than to discordant displays. In contrast, the effect of concordance was independent of the specific combination of movement and song types in a display. Our results indicate that the concordance of movements and songs was critical to the efficacy of the display and suggest that the information that birds gained from concordance could not be obtained by adding information from movements and songs.

Ripple effects in a communication network: anti-eavesdropper defence elicits elaborated sexual signals in rival males

Emitting conspicuous signals into the environment to attract mates comes with the increased risk of interception by eavesdropping enemies. As a defence, a commonly described strategy is for signallers to group together in leks, diluting each individual's risk. Lekking systems are often highly social settings in which competing males dynamically alter their signalling behaviour to attract mates. Thus, signalling at the lek requires navigating fluctuations in risk, competition and reproductive opportunities. Here, we investigate how behavioural defence strategies directed at an eavesdropping enemy have cascading effects across the communication network. We investigated these behaviours in the túngara frog (Engystomops pustulosus), examining how a calling male's swatting defence directed at frog-biting midges indirectly affects the calling behaviour of his rival. We found that the rival responds to swat-induced water ripples by increasing his call rate and complexity. Then, performing phonotaxis experiments, we found that eavesdropping fringe-lipped bats (Trachops cirrhosus) do not exhibit a preference for a swatting male compared to his rival, but females strongly prefer the rival male. Defences to minimize attacks from eavesdroppers thus shift the mate competition landscape in favour of rival males. By modulating the attractiveness of signalling prey to female receivers, we posit that eavesdropping micropredators likely have an unappreciated impact on the ecology and evolution of sexual communication systems.

The quality of avian vocal duets can be assessed independently of the spatial separation of signallers

Interactions among groups are often mediated through signals, including coordinated calls such as duets, and the degree of temporal coordination within a group can affect signal efficacy. However, in addition to intrinsic duet quality, the spatial arrangement of callers also affects the timing of calls. So, can listeners discriminate temporal effects caused by intrinsic duet quality compared to spatial arrangement? Such discrimination would allow assessment of quality of duets produced by a pair, as distinct from transient extrinsic spatial effects. To address this issue, we studied experimentally the influence of intrinsic duet quality and spatial arrangement on the efficacy of Australian magpie-lark (Grallina cyanoleuca) vocal duets. Breeding pairs duet at varying distances from each other and to multiple neighbours. Coordinated duets are more effective territorial signals than uncoordinated duets, but it remains unclear whether listeners can discriminate the effects of quality and spatial arrangement. Our playback experiment showed that any deviation from perfect regularity of partners' notes reduced duet efficacy, but that lack of coordination due to spatial separation (slower tempo and offset of notes) had a lower effect on efficacy than effects due to intrinsic quality (irregularity). Our results therefore provide experimental evidence that the temporal organisation of group vocalisations could signal coalition quality independently of spatial effects.

Reality and illusion: the assessment of angular separation of multi-modal signallers in a duetting bird

The spatial distribution of cooperating individuals plays a strategic role in territorial interactions of many group-living animals, and can indicate group cohesion. Vocalizations are commonly used to judge the distribution of signallers, but the spatial resolution of sounds is poor. Many species therefore accompany calls with movement; however, little is known about the role of audio-visual perception in natural interactions. We studied the effect of angular separation on the efficacy of multimodal duets in the Australian magpie-lark, Grallina cyanoleuca. We tested specifically whether conspicuous wing movements, which typically accompany duets, affect responses to auditory angular separation. Multimodal playbacks of duets using robotic models and speakers showed that birds relied primarily on acoustic cues when visual and auditory angular separations were congruent, but used both modalities to judge separation between the signallers when modalities were spatially incongruent. The visual component modified the effect of acoustic separation: robotic models that were apart weakened the response when speakers were together, while models that were together strengthened responses when speakers were apart. Our results show that responses are stronger when signallers are together, and suggest that males were are able to bind information cross-modally on the senders' spatial location, which is consistent with a multisensory illusion.

Parasite defensive limb movements enhance acoustic signal attraction in male little torrent frogs

Many animals rely on complex signals that target multiple senses to attract mates and repel rivals. These multimodal displays can however also attract unintended receivers, which can be an important driver of signal complexity. Despite being taxonomically widespread, we often lack insight into how multimodal signals evolve from unimodal signals and in particular what roles unintended eavesdroppers play. Here, we assess whether the physical movements of parasite defense behavior increase the complexity and attractiveness of an acoustic sexual signal in the little torrent frog (Amolops torrentis). Calling males of this species often display limb movements in order to defend against blood-sucking parasites such as frog-biting midges that eavesdrop on their acoustic signal. Through mate choice tests we show that some of these midge-evoked movements influence female preference for acoustic signals. Our data suggest that midge-induced movements may be incorporated into a sexual display, targeting both hearing and vision in the intended receiver. Females may play an important role in incorporating these multiple components because they prefer signals which combine multiple modalities. Our results thus help to understand the relationship between natural and sexual selection pressure operating on signalers and how in turn this may influence multimodal signal evolution.

Training-induced plasticity enables visualizing sounds with a visual-to-auditory conversion device

Sensory substitution devices aim at restoring visual functions by converting visual information into auditory or tactile stimuli. Although these devices show promise in the range of behavioral abilities they allow, the processes underlying their use remain underspecified. In particular, while an initial debate focused on the visual versus auditory or tactile nature of sensory substitution, since over a decade, the idea that it reflects a mixture of both has emerged. In order to investigate behaviorally the extent to which visual and auditory processes are involved, participants completed a Stroop-like crossmodal interference paradigm before and after being trained with a conversion device which translates visual images into sounds. In addition, participants' auditory abilities and their phenomenologies were measured. Our study revealed that, after training, when asked to identify sounds, processes shared with vision were involved, as participants’ performance in sound identification was influenced by the simultaneously presented visual distractors. In addition, participants’ performance during training and their associated phenomenology depended on their auditory abilities, revealing that processing finds its roots in the input sensory modality. Our results pave the way for improving the design and learning of these devices by taking into account inter-individual differences in auditory and visual perceptual strategies.

Covariation among multimodal components in the courtship display of the túngara frog

Communication systems often include a variety of components, including those that span modalities, which may facilitate detection and decision-making. For example, female túngara frogs and fringe-lipped bats generally rely on acoustic mating signals to find male túngara frogs in a mating or foraging context, respectively. However, two additional cues (vocal sac inflation and water ripples) can enhance detection and choice behavior. To date, we do not know the natural variation and covariation of these three components. To address this, we made detailed recordings of calling males, including call amplitude, vocal sac volume and water ripple height, in 54 frogs (2430 calls). We found that all three measures correlated, with the strongest association between the vocal sac volume and call amplitude. We also found that multimodal models predicted the mass of calling males better than unimodal models. These results demonstrate how multimodal components of a communication system relate to each other and provide an important foundation for future studies on how receivers integrate and compare complex displays.

Conspecific odor cues induce different vocal responses in serrate-legged small treefrogs, but only in the absence of acoustic signals

Background

Signal detection is crucial to survival and successful reproduction, and animals often modify behavioral decisions based on information they obtained from the social context. Undeniably, the decision-making in male-male competition and female choice of anurans (frogs and toads) depends heavily on acoustic signals. However, increasing empirical evidence suggests that additional or alternative types of cue (e.g., visual, chemical, and vibratory) can be used to detect, discriminate and locate conspecifics in many anuran species. Nevertheless, few studies have investigated whether conspecific odor cues affect male’s calling behavior. In this study, we conducted an experiment to investigate whether and how different chemical cues (male odors, female odors, and stress odors) from conspecifics affect male’s calling strategies in serrate-legged small treefrogs (Kurixalus odontotarsus), and whether the combined chemical and acoustic stimuli have additive effects on calling behavior or not.

Results

We found that compared with female odors, male K. odontotarsus reduced calling investment in response to male odors or stress odors, in the absence of rival’s advertisement calls. When odor stimuli and advertisement calls were presented simultaneously, however, there were no differences in the vocal response of focal males among odor groups.

Conclusions

These results provide evidence that male treefrogs switch calling investment according to different odor cues from conspecifics, and further demonstrate that calling behavior can be affected by chemical cues in anuran species. Our study highlights the potential role of airborne chemical cues in sex identification and contributes to increase our understanding of anuran communication.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12983-021-00415-y.

Multisensory modalities increase working memory for mating signals in a treefrog

Animal choruses, such as those found in insects and frogs, are often intermittent. Thus, females sampling males in the chorus might have to remember the location of the potential mates' calls during periods of silence. Although a number of studies have shown that frogs use and prefer multimodal mating signals, usually acoustic plus visual, it is not clear why they do so. Here we tested the hypothesis that preference for multimodal signals over unimodal signals might be due to multimodal signals instantiating longer memories than unimodal signals, particularly during the inter-chorus intervals. We tested this hypothesis in serrate-legged small treefrogs Kurixalus odontotarsus whose males produce advertisement calls accompanied by conspicuous vocal sac inflation. Females were tested with acoustic and acoustic + visual (video of inflating-deflating vocal sac) mating calls. We found that females prefer multimodal calls over unimodal, audio-only calls. Furthermore, multimodal calls are still preferred after a silent period of up to 30 s, a time that spans the average silent period of the chorus. This was not true of unimodal calls. Our results demonstrate that a multimodal signal can engage longer working memory than a unimodal signal, and thus female memory might favour the evolution of multimodal signals in males through sexual selection. Selection might also favour female preference for multimodal signals if longer memory facilitates mate searching and assessment. Our study does not allow us to elucidate the sequence of evolution of this trait and preference.

Male serrate-legged treefrogs adjust competition strategies according to visual or chemical cues from females

There is increasing evidence that many anurans use multimodal cues to detect, discriminate and/or locate conspecifics and thus modify their behaviors. To date, however, most studies have focused on the roles of multimodal cues in female choice or male-male interactions. In the present study, we conducted an experiment to investigate whether male serrate-legged small treefrogs (Kurixalus odontotarsus) used visual or chemical cues to detect females and thus altered their competition strategies in different calling contexts. Three acoustic stimuli (advertisement calls, aggressive calls and compound calls) were broadcast in a randomized order after a spontaneous period to focal males in one of four treatment groups: combined visual and chemical cues of a female, only chemical cues, only visual cues and a control (with no females). We recorded the vocal responses of the focal males during each 3 min period. Our results demonstrate that males reduce the total number of calls in response to the presence of females, regardless of how they perceived the females. In response to advertisement calls and compound calls, males that perceived females through chemical cues produced relatively fewer advertisement calls but more aggressive calls. In addition, they produced relatively more aggressive calls during the playback of aggressive calls. Taken together, our study suggests that male Kodontotarsus adjust their competition strategies according to the visual or chemical cues of potential mates and highlights the important role of multisensory cues in male frogs' perception of females.

Environmental constraints on size-dependent signaling affects mating and rival interactions

Advertisement signals can convey information about a sender’s characteristics, such as body size. The reliability of signals, however, can be reduced when signal production is partially dependent on the environment. Here, we assess the effect of display-site properties on the production, attractiveness and honesty of sexual signals. We recorded male túngara frogs (Physalaemus pustulosus) while manipulating water levels in order to constrain calling. We found that water level affected male call properties in a size-dependent manner, with call amplitude being less affected in smaller males when forced to call in shallow water. Next, we tested how size-dependent and display-site-dependent signaling affected female choice and rival competition. Both males and females showed the strongest response to the call of a large male when he was calling at the deepest water levels. However, females showed no preference for large over small males when both were recorded in shallow water levels, or, depending on the call rate and timing of calls, even preferred small males. Likewise, males responded equally to large and small rivals recorded calling during shallow water level trials. Our experiments show that display-site properties can influence signal production and attractiveness in a size-dependent manner. These results can have important consequences for the evolution of signaling, as small males may be able to use their size to their advantage when selecting appropriate display sites and thereby outcompete large males.

Perceived Synchrony of Frog Multimodal Signal Components Is Influenced by Content and Order

Multimodal signaling is common in communication systems. Depending on the species, individual signal components may be produced synchronously as a result of physiological constraint (fixed) or each component may be produced independently (fluid) in time. For animals that rely on fixed signals, a basic prediction is that asynchrony between the components should degrade the perception of signal salience, reducing receiver response. Male túngara frogs, Physalaemus pustulosus, produce a fixed multisensory courtship signal by vocalizing with two call components (whines and chucks) and inflating a vocal sac (visual component). Using a robotic frog, we tested female responses to variation in the temporal arrangement between acoustic and visual components. When the visual component lagged a complex call (whine + chuck), females largely rejected this asynchronous multisensory signal in favor of the complex call absent the visual cue. When the chuck component was removed from one call, but the robofrog inflation lagged the complex call, females responded strongly to the asynchronous multimodal signal. When the chuck component was removed from both calls, females reversed preference and responded positively to the asynchronous multisensory signal. When the visual component preceded the call, females responded as often to the multimodal signal as to the call alone. These data show that asynchrony of a normally fixed signal does reduce receiver responsiveness. The magnitude and overall response, however, depend on specific temporal interactions between the acoustic and visual components. The sensitivity of túngara frogs to lagging visual cues, but not leading ones, and the influence of acoustic signal content on the perception of visual asynchrony is similar to those reported in human psychophysics literature. Virtually all acoustically communicating animals must conduct auditory scene analyses and identify the source of signals. Our data suggest that some basic audiovisual neural integration processes may be at work in the vertebrate brain.

Sometimes noise is beneficial: stream noise informs vocal communication in the little torrent frog Amolops torrentis

Many kinds of environmental noise can interfere with acoustic communication and efficient decision making in terrestrial species. Here we identified an exception to this generalization in a streamside species, the little torrent frog (Amolops torrentis) which communicates in a stream noise environment. To determine whether stream noise can act as a cue regarding the microhabitat characteristics of senders, we performed phonotaxis experiments using stimulus pairs constructed with synthetic male calls (high or low dominant frequency) and stream noise with varied signal-to-noise ratios. We found that females prefer calls with high amplitude stream noise added compared to those with low amplitude stream noise added for both high and low dominant frequency stimulus pairs; however, stream noise itself was not attractive in the absence of calls. These results show that stream noise can function as a cue that may be used by females for enhancing the attractiveness of calls. Stream noise associates closely with rocks, topographies and vegetation and may thus provide useful microhabitat information for signal receivers, thereby acting on sexual selection. These data therefore contribute to our understanding of how the perception of mate attractiveness in heterogeneous ecological environments can evolve.

Multimodal cues improve prey localization under complex environmental conditions

Predators often eavesdrop on sexual displays of their prey. These displays can provide multimodal cues that aid predators, but the benefits in attending to them should depend on the environmental sensory conditions under which they forage. We assessed whether bats hunting for frogs use multimodal cues to locate their prey and whether their use varies with ambient conditions. We used a robotic set-up mimicking the sexual display of a male túngara frog (Physalaemus pustulosus) to test prey assessment by fringe-lipped bats (Trachops cirrhosus). These predatory bats primarily use sound of the frog's call to find their prey, but the bats also use echolocation cues returning from the frog's dynamically moving vocal sac. In the first experiment, we show that multimodal cues affect attack behaviour: bats made narrower flank attack angles on multimodal trials compared with unimodal trials during which they could only rely on the sound of the frog. In the second experiment, we explored the bat's use of prey cues in an acoustically more complex environment. Túngara frogs often form mixed-species choruses with other frogs, including the hourglass frog (Dendropsophus ebraccatus). Using a multi-speaker set-up, we tested bat approaches and attacks on the robofrog under three different levels of acoustic complexity: no calling D. ebraccatus males, two calling D. ebraccatus males and five D. ebraccatus males. We found that bats are more directional in their approach to the robofrog when more D. ebraccatus males were calling. Thus, bats seemed to benefit more from multimodal cues when confronted with increased levels of acoustic complexity in their foraging environments. Our data have important consequences for our understanding of the evolution of multimodal sexual displays as they reveal how environmental conditions can alter the natural selection pressures acting on them.

Wind- and Rain-Induced Vibrations Impose Different Selection Pressures on Multimodal Signaling

The world is a noisy place, and animals have evolved a myriad of strategies to communicate in it. Animal communication signals are, however, often multimodal; their components can be processed by multiple sensory systems, and noise can thus affect signal components across different modalities. We studied the effect of environmental noise on multimodal communication in the túngara frog (Physalaemus pustulosus). Males communicate with rivals using airborne sounds combined with call-induced water ripples. We tested males under control as well as noisy conditions in which we mimicked rain- and wind-induced vibrations on the water surface. Males responded more strongly to a multimodal playback in which sound and ripples were combined, compared to a unimodal sound-only playback, but only in the absence of rain and wind. Under windy conditions, males decreased their response to the multimodal playback, suggesting that wind noise interferes with the detection of rival ripples. Under rainy conditions, males increased their response, irrespective of signal playback, suggesting that different noise sources can have different impacts on communication. Our findings show that noise in an additional sensory channel can affect multimodal signal perception and thereby drive signal evolution, but not always in the expected direction.

A systems approach to animal communication

Why animal communication displays are so complex and how they have evolved are active foci of research with a long and rich history. Progress towards an evolutionary analysis of signal complexity, however, has been constrained by a lack of hypotheses to explain similarities and/or differences in signalling systems across taxa. To address this, we advocate incorporating a systems approach into studies of animal communication--an approach that includes comprehensive experimental designs and data collection in combination with the implementation of systems concepts and tools. A systems approach evaluates overall display architecture, including how components interact to alter function, and how function varies in different states of the system. We provide a brief overview of the current state of the field, including a focus on select studies that highlight the dynamic nature of animal signalling. We then introduce core concepts from systems biology (redundancy, degeneracy, pluripotentiality, and modularity) and discuss their relationships with system properties (e.g. robustness, flexibility, evolvability). We translate systems concepts into an animal communication framework and accentuate their utility through a case study. Finally, we demonstrate how consideration of the system-level organization of animal communication poses new practical research questions that will aid our understanding of how and why animal displays are so complex.

Pollution going multimodal: the complex impact of the human-altered sensory environment on animal perception and performance

Anthropogenic sensory pollution is affecting ecosystems worldwide. Human actions generate acoustic noise, emanate artificial light and emit chemical substances. All of these pollutants are known to affect animals. Most studies on anthropogenic pollution address the impact of pollutants in unimodal sensory domains. High levels of anthropogenic noise, for example, have been shown to interfere with acoustic signals and cues. However, animals rely on multiple senses, and pollutants often co-occur. Thus, a full ecological assessment of the impact of anthropogenic activities requires a multimodal approach. We describe how sensory pollutants can co-occur and how covariance among pollutants may differ from natural situations. We review how animals combine information that arrives at their sensory systems through different modalities and outline how sensory conditions can interfere with multimodal perception. Finally, we describe how sensory pollutants can affect the perception, behaviour and endocrinology of animals within and across sensory modalities. We conclude that sensory pollution can affect animals in complex ways due to interactions among sensory stimuli, neural processing and behavioural and endocrinal feedback. We call for more empirical data on covariance among sensory conditions, for instance, data on correlated levels in noise and light pollution. Furthermore, we encourage researchers to test animal responses to a full-factorial set of sensory pollutants in the presence or the absence of ecologically important signals and cues. We realize that such approach is often time and energy consuming, but we think this is the only way to fully understand the multimodal impact of sensory pollution on animal performance and perception.

Sophisticated Communication in the Brazilian Torrent Frog Hylodes japi

Intraspecific communication in frogs plays an important role in the recognition of conspecifics in general and of potential rivals or mates in particular and therefore with relevant consequences for pre-zygotic reproductive isolation. We investigate intraspecific communication in Hylodes japi, an endemic Brazilian torrent frog with territorial males and an elaborate courtship behavior. We describe its repertoire of acoustic signals as well as one of the most complex repertoires of visual displays known in anurans, including five new visual displays. Previously unknown in frogs, we also describe a bimodal inter-sexual communication system where the female stimulates the male to emit a courtship call. As another novelty for frogs, we show that in addition to choosing which limb to signal with, males choose which of their two vocal sacs will be used for visual signaling. We explain how and why this is accomplished. Control of inflation also provides additional evidence that vocal sac movement and color must be important for visual communication, even while producing sound. Through the current knowledge on visual signaling in Neotropical torrent frogs (i.e. hylodids), we discuss and highlight the behavioral diversity in the family Hylodidae. Our findings indicate that communication in species of Hylodes is undoubtedly more sophisticated than we expected and that visual communication in anurans is more widespread than previously thought. This is especially true in tropical regions, most likely due to the higher number of species and phylogenetic groups and/or to ecological factors, such as higher microhabitat diversity.

Signal Diversity, Sexual Selection, and Speciation

Communication is ubiquitous. Developing a framework for the diversity of signals has important consequences for understanding alternative models of sexual selection and the processes contributing to speciation. In this article we review how models of neutral evolution in the perceptual space of signal perceivers provide a first step toward constructing a framework for signal diversity. We discuss how the distinction between additive and multiplicative effects of multimodal signaling represents a second step. We then assess how signal efficiency, reliability, and the aesthetics of perceivers provide distinct mechanisms for signals to be effective, thereby partly explaining signal diversity. Understanding the relative contribution of each of these mechanisms to the effectiveness of mate choice signals unravels the relative importance of alternative models of sexual selection. It can also help to distinguish whether divergence of communication is a driver or a consequence of speciation. Throughout the review we emphasize the importance of verification and learning in repeated interactions for understanding variation in signals.

Aggressive Bimodal Communication in Domestic Dogs, Canis familiaris

Evidence of animal multimodal signalling is widespread and compelling. Dogs’ aggressive vocalisations (growls and barks) have been extensively studied, but without any consideration of the simultaneously produced visual displays. In this study we aimed to categorize dogs’ bimodal aggressive signals according to the redundant/non-redundant classification framework. We presented dogs with unimodal (audio or visual) or bimodal (audio-visual) stimuli and measured their gazing and motor behaviours. Responses did not qualitatively differ between the bimodal and two unimodal contexts, indicating that acoustic and visual signals provide redundant information. We could not further classify the signal as ‘equivalent’ or ‘enhancing’ as we found evidence for both subcategories. We discuss our findings in relation to the complex signal framework, and propose several hypotheses for this signal’s function.