Soundscapes predict species occurrence in tropical forests

Unlocking the soundscape of coral reefs with artificial intelligence: pretrained networks and unsupervised learning win out

Passive acoustic monitoring can offer insights into the state of coral reef ecosystems at low-costs and over extended temporal periods. Comparison of whole soundscape properties can rapidly deliver broad insights from acoustic data, in contrast to detailed but time-consuming analysis of individual bioacoustic events. However, a lack of effective automated analysis for whole soundscape data has impeded progress in this field. Here, we show that machine learning (ML) can be used to unlock greater insights from reef soundscapes. We showcase this on a diverse set of tasks using three biogeographically independent datasets, each containing fish community (high or low), coral cover (high or low) or depth zone (shallow or mesophotic) classes. We show supervised learning can be used to train models that can identify ecological classes and individual sites from whole soundscapes. However, we report unsupervised clustering achieves this whilst providing a more detailed understanding of ecological and site groupings within soundscape data. We also compare three different approaches for extracting feature embeddings from soundscape recordings for input into ML algorithms: acoustic indices commonly used by soundscape ecologists, a pretrained convolutional neural network (P-CNN) trained on 5.2 million hrs of YouTube audio, and CNN's which were trained on each individual task (T-CNN). Although the T-CNN performs marginally better across tasks, we reveal that the P-CNN offers a powerful tool for generating insights from marine soundscape data as it requires orders of magnitude less computational resources whilst achieving near comparable performance to the T-CNN, with significant performance improvements over the acoustic indices. Our findings have implications for soundscape ecology in any habitat.

Sound evidence for biodiversity monitoring

Bioacoustics and artificial intelligence facilitate ecological studies of animal populations.

Global birdsong embeddings enable superior transfer learning for bioacoustic classification

Automated bioacoustic analysis aids understanding and protection of both marine and terrestrial animals and their habitats across extensive spatiotemporal scales, and typically involves analyzing vast collections of acoustic data. With the advent of deep learning models, classification of important signals from these datasets has markedly improved. These models power critical data analyses for research and decision-making in biodiversity monitoring, animal behaviour studies, and natural resource management. However, deep learning models are often data-hungry and require a significant amount of labeled training data to perform well. While sufficient training data is available for certain taxonomic groups (e.g., common bird species), many classes (such as rare and endangered species, many non-bird taxa, and call-type) lack enough data to train a robust model from scratch. This study investigates the utility of feature embeddings extracted from audio classification models to identify bioacoustic classes other than the ones these models were originally trained on. We evaluate models on diverse datasets, including different bird calls and dialect types, bat calls, marine mammals calls, and amphibians calls. The embeddings extracted from the models trained on bird vocalization data consistently allowed higher quality classification than the embeddings trained on general audio datasets. The results of this study indicate that high-quality feature embeddings from large-scale acoustic bird classifiers can be harnessed for few-shot transfer learning, enabling the learning of new classes from a limited quantity of training data. Our findings reveal the potential for efficient analyses of novel bioacoustic tasks, even in scenarios where available training data is limited to a few samples.

Soundscapes and deep learning enable tracking biodiversity recovery in tropical forests

Tropical forest recovery is fundamental to addressing the intertwined climate and biodiversity loss crises. While regenerating trees sequester carbon relatively quickly, the pace of biodiversity recovery remains contentious. Here, we use bioacoustics and metabarcoding to measure forest recovery post-agriculture in a global biodiversity hotspot in Ecuador. We show that the community composition, and not species richness, of vocalizing vertebrates identified by experts reflects the restoration gradient. Two automated measures - an acoustic index model and a bird community composition derived from an independently developed Convolutional Neural Network - correlated well with restoration (adj-R² = 0.62 and 0.69, respectively). Importantly, both measures reflected composition of non-vocalizing nocturnal insects identified via metabarcoding. We show that such automated monitoring tools, based on new technologies, can effectively monitor the success of forest recovery, using robust and reproducible data.

Limits to the accurate and generalizable use of soundscapes to monitor biodiversity

Although eco-acoustic monitoring has the potential to deliver biodiversity insight on vast scales, existing analytical approaches behave unpredictably across studies. We collated 8,023 audio recordings with paired manual avifaunal point counts to investigate whether soundscapes could be used to monitor biodiversity across diverse ecosystems. We found that neither univariate indices nor machine learning models were predictive of species richness across datasets but soundscape change was consistently indicative of community change. Our findings indicate that there are no common features of biodiverse soundscapes and that soundscape monitoring should be used cautiously and in conjunction with more reliable in-person ecological surveys.

A workflow for the automated detection and classification of female gibbon calls from long-term acoustic recordings

Passive acoustic monitoring (PAM) allows for the study of vocal animals on temporal and spatial scales difficult to achieve using only human observers. Recent improvements in recording technology, data storage, and battery capacity have led to increased use of PAM. One of the main obstacles in implementing wide-scale PAM programs is the lack of open-source programs that efficiently process terabytes of sound recordings and do not require large amounts of training data. Here we describe a workflow for detecting, classifying, and visualizing female Northern grey gibbon calls in Sabah, Malaysia. Our approach detects sound events using band-limited energy summation and does binary classification of these events (gibbon female or not) using machine learning algorithms (support vector machine and random forest). We then applied an unsupervised approach (affinity propagation clustering) to see if we could further differentiate between true and false positives or the number of gibbon females in our dataset. We used this workflow to address three questions: (1) does this automated approach provide reliable estimates of temporal patterns of gibbon calling activity; (2) can unsupervised approaches be applied as a post-processing step to improve the performance of the system; and (3) can unsupervised approaches be used to estimate how many female individuals (or clusters) there are in our study area? We found that performance plateaued with >160 clips of training data for each of our two classes. Using optimized settings, our automated approach achieved a satisfactory performance (F1 score ~ 80%). The unsupervised approach did not effectively differentiate between true and false positives or return clusters that appear to correspond to the number of females in our study area. Our results indicate that more work needs to be done before unsupervised approaches can be reliably used to estimate the number of individual animals occupying an area from PAM data. Future work applying these methods across sites and different gibbon species and comparisons to deep learning approaches will be crucial for future gibbon conservation initiatives across Southeast Asia.

Acoustic indices as proxies for biodiversity: a meta-analysis

As biodiversity decreases worldwide, the development of effective techniques to track changes in ecological communities becomes an urgent challenge. Together with other emerging methods in ecology, acoustic indices are increasingly being used as novel tools for rapid biodiversity assessment. These indices are based on mathematical formulae that summarise the acoustic features of audio samples, with the aim of extracting meaningful ecological information from soundscapes. However, the application of this automated method has revealed conflicting results across the literature, with conceptual and empirical controversies regarding its primary assumption: a correlation between acoustic and biological diversity. After more than a decade of research, we still lack a statistically informed synthesis of the power of acoustic indices that elucidates whether they effectively function as proxies for biological diversity. Here, we reviewed studies testing the relationship between diversity metrics (species abundance, species richness, species diversity, abundance of sounds, and diversity of sounds) and the 11 most commonly used acoustic indices. From 34 studies, we extracted 364 effect sizes that quantified the magnitude of the direct link between acoustic and biological estimates and conducted a meta-analysis. Overall, acoustic indices had a moderate positive relationship with the diversity metrics (r = 0.33, CI [0.23, 0.43]), and showed an inconsistent performance, with highly variable effect sizes both within and among studies. Over time, studies have been increasingly disregarding the validation of the acoustic estimates and those examining this link have been progressively reporting smaller effect sizes. Some of the studied indices [acoustic entropy index (H), normalised difference soundscape index (NDSI), and acoustic complexity index (ACI)] performed better in retrieving biological information, with abundance of sounds (number of sounds from identified or unidentified species) being the best estimated diversity facet of local communities. We found no effect of the type of monitored environment (terrestrial versus aquatic) and the procedure for extracting biological information (acoustic versus non-acoustic) on the performance of acoustic indices, suggesting certain potential to generalise their application across research contexts. We also identified common statistical issues and knowledge gaps that remain to be addressed in future research, such as a high rate of pseudoreplication and multiple unexplored combinations of metrics, taxa, and regions. Our findings confirm the limitations of acoustic indices to efficiently quantify alpha biodiversity and highlight that caution is necessary when using them as surrogates of diversity metrics, especially if employed as single predictors. Although these tools are able partially to capture changes in diversity metrics, endorsing to some extent the rationale behind acoustic indices and suggesting them as promising bases for future developments, they are far from being direct proxies for biodiversity. To guide more efficient use and future research, we review their principal theoretical and practical shortcomings, as well as prospects and challenges of acoustic indices in biodiversity assessment. Altogether, we provide the first comprehensive and statistically based overview on the relation between acoustic indices and biodiversity and pave the way for a more standardised and informed application for biodiversity monitoring.

Computational bioacoustics with deep learning: a review and roadmap

Animal vocalisations and natural soundscapes are fascinating objects of study, and contain valuable evidence about animal behaviours, populations and ecosystems. They are studied in bioacoustics and ecoacoustics, with signal processing and analysis an important component. Computational bioacoustics has accelerated in recent decades due to the growth of affordable digital sound recording devices, and to huge progress in informatics such as big data, signal processing and machine learning. Methods are inherited from the wider field of deep learning, including speech and image processing. However, the tasks, demands and data characteristics are often different from those addressed in speech or music analysis. There remain unsolved problems, and tasks for which evidence is surely present in many acoustic signals, but not yet realised. In this paper I perform a review of the state of the art in deep learning for computational bioacoustics, aiming to clarify key concepts and identify and analyse knowledge gaps. Based on this, I offer a subjective but principled roadmap for computational bioacoustics with deep learning: topics that the community should aim to address, in order to make the most of future developments in AI and informatics, and to use audio data in answering zoological and ecological questions.