Using deep neural networks to model similarity between visual patterns: Application to fish sexual signals

Dorsoventral comparison of intraspecific variation in the butterfly wing pattern using a convolutional neural network

Butterfly wing patterns exhibit notable differences between the dorsal and ventral surfaces, and morphological analyses of them have provided insights into the ecological and behavioural characteristics of wing patterns. Conventional methods for dorsoventral comparisons are constrained by the need for homologous patches or shared features between two surfaces, limiting their applicability across species. We used a convolutional neural network (CNN)-based analysis, which can compare images of the two surfaces without focusing on homologous patches or features, to detect dorsoventral bias in two types of intraspecific variation: sexual dimorphism and mimetic polymorphism. Using specimen images of 29 species, we first showed that the level of sexual dimorphism calculated by CNN-based analysis corresponded well with traditional assessments of sexual dissimilarity, demonstrating the validity of the method. Dorsal biases were widely detected in sexual dimorphism, suggesting that the conventional hypothesis of dorsally biased sexual selection can be supported in a broader range of species. In contrast, mimetic polymorphism showed no such bias, indicating the importance of both surfaces in mimicry. Our study demonstrates the potential versatility of CNN in comparing wing patterns between the two surfaces, while elucidating the relationship between dorsoventrally different selections and dorsoventral biases in intraspecific variations.

Male and female contributions to diversity among birdwing butterfly images

Machine learning (ML) newly enables tests for higher inter-species diversity in visible phenotype (disparity) among males versus females, predictions made from Darwinian sexual selection versus Wallacean natural selection, respectively. Here, we use ML to quantify variation across a sample of > 16,000 dorsal and ventral photographs of the sexually dimorphic birdwing butterflies (Lepidoptera: Papilionidae). Validation of image embedding distances, learnt by a triplet-trained, deep convolutional neural network, shows ML can be used for automated reconstruction of phenotypic evolution achieving measures of phylogenetic congruence to genetic species trees within a range sampled among genetic trees themselves. Quantification of sexual disparity difference (male versus female embedding distance), shows sexually and phylogenetically variable inter-species disparity. Ornithoptera exemplify high embedded male image disparity, diversification of selective optima in fitted multi-peak OU models and accelerated divergence, with cases of extreme divergence in allopatry and sympatry. However, genus Troides shows inverted patterns, including comparatively static male embedded phenotype, and higher female than male disparity - though within an inferred selective regime common to these females. Birdwing shapes and colour patterns that are most phenotypically distinctive in ML similarity are generally those of males. However, either sex can contribute majoritively to observed phenotypic diversity among species.

Using Artificial Intelligence to Analyze Non-Human Drawings: A First Step with Orangutan Productions

Drawings have been widely used as a window to the mind; as such, they can reveal some aspects of the cognitive and emotional worlds of other animals that can produce them. The study of non-human drawings, however, is limited by human perception, which can bias the methodology and interpretation of the results. Artificial intelligence can circumvent this issue by allowing automated, objective selection of features used to analyze drawings. In this study, we use artificial intelligence to investigate seasonal variations in drawings made by Molly, a female orangutan who produced more than 1299 drawings between 2006 and 2011 at the Tama Zoological Park in Japan. We train the VGG19 model to first classify the drawings according to the season in which they are produced. The results show that deep learning is able to identify subtle but significant seasonal variations in Molly's drawings, with a classification accuracy of 41.6%. We use VGG19 to investigate the features that influence this seasonal variation. We analyze separate features, both simple and complex, related to color and patterning, and to drawing content and style. Content and style classification show maximum performance for moderately complex, highly complex, and holistic features, respectively. We also show that both color and patterning drive seasonal variation, with the latter being more important than the former. This study demonstrates how deep learning can be used to objectively analyze non-figurative drawings and calls for applications to non-primate species and scribbles made by human toddlers.