The science and technology of kinematic measurements in a century of Journal of Experimental Biology

Towards the yin and yang of fish locomotion: linking energetics, ecology and mechanics through field and lab approaches

Most of our understanding of fish locomotion has focused on elementary behaviors such as steady swimming and escape responses in simple environments. As the field matures, increasing attention is being paid to transient and unsteady behaviors that characterize more complex interactions with the environment. This Commentary advocates for an ecologically relevant approach to lab studies. Specific examples have brought new understanding to the energetic consequences of fish swimming, such as (1) station holding around bluff bodies, which departs drastically from steady swimming in almost all aspects of kinematics, muscle activity and energetics, and (2) transient behaviors such as acceleration and feeding, which are critical to survival but often neglected because of challenges in measuring costs. Beyond the lab, a far richer diversity of behaviors is available when fish are given enough space and time to move. Mesocosm studies are poised to reveal new insights into fish swimming that are inaccessible in laboratory settings. Next-generation biologgers that incorporate neural recordings will usher in a new era for understanding biomechanics in the wild and open the door for a more mechanistic understanding of how changing environments affect animal movement. These advances promise to allow insights into animal locomotion in ways that will mutually complement and accelerate laboratory and field studies in the years to come.

Modulation of prey capture kinematics in relation to prey distance helps predict success

Predators are not perfect, as some of their prey capture attempts result in failure. Successful attempts may be partly due to predators modulating their capture kinematics in relation to variation in the visual cues of the prey to increase the probability of success. In praying mantises, which have been suggested to possess stereoscopic vision, variation in prey distance has been shown to elicit variation in the probability of an attempt. However, it remains to be examined whether variation in prey distance results in mantises modulating their attempt to successfully capture prey. The goals of this study were to examine these relationships using the praying mantis system. Using 11 adult female Sphodromantis lineola, we recorded 192 prey capture attempts at 1000 Hz with two cameras to examine the 3D kinematics of successful and unsuccessful prey capture attempts. Using a combination of principal component analysis (PCA) and logistic regression, our results show that as prey distance increases, mantises adjust through greater and faster expansion of the forelegs and body (PC1), which significantly predicts capture success. However, PC1 only explains 22% of the variation in all prey capture attempts, suggesting that the other components may be related to additional aspects of the prey. Our results suggest that the distances at which mantises prefer to attempt to capture prey may be the result of their greater probability of successfully capturing the prey. These results highlight the range of motions mantises use when attempting to capture prey, suggesting flexibility in their prey capture attempts in relation to prey position.

Application of a novel deep learning-based 3D videography workflow to bat flight

Studying the detailed biomechanics of flying animals requires accurate three-dimensional coordinates for key anatomical landmarks. Traditionally, this relies on manually digitizing animal videos, a labor-intensive task that scales poorly with increasing framerates and numbers of cameras. Here, we present a workflow that combines deep learning-powered automatic digitization with filtering and correction of mislabeled points using quality metrics from deep learning and 3D reconstruction. We tested our workflow using a particularly challenging scenario: bat flight. First, we documented four bats flying steadily in a 2 m3 wind tunnel test section. Wing kinematic parameters resulting from manually digitizing bats with markers applied to anatomical landmarks were not significantly different from those resulting from applying our workflow to the same bats without markers for five out of six parameters. Second, we compared coordinates from manual digitization against those yielded via our workflow for bats flying freely in a 344 m3 enclosure. Average distance between coordinates from our workflow and those from manual digitization was less than a millimeter larger than the average human-to-human coordinate distance. The improved efficiency of our workflow has the potential to increase the scalability of studies on animal flight biomechanics.

Through the looking glass: attempting to predict future opportunities and challenges in experimental biology

To celebrate its centenary year, Journal of Experimental Biology (JEB) commissioned a collection of articles examining the past, present and future of experimental biology. This Commentary closes the collection by considering the important research opportunities and challenges that await us in the future. We expect that researchers will harness the power of technological advances, such as '-omics' and gene editing, to probe resistance and resilience to environmental change as well as other organismal responses. The capacity to handle large data sets will allow high-resolution data to be collected for individual animals and to understand population, species and community responses. The availability of large data sets will also place greater emphasis on approaches such as modeling and simulations. Finally, the increasing sophistication of biologgers will allow more comprehensive data to be collected for individual animals in the wild. Collectively, these approaches will provide an unprecedented understanding of 'how animals work' as well as keys to safeguarding animals at a time when anthropogenic activities are degrading the natural environment.

replicAnt: a pipeline for generating annotated images of animals in complex environments using Unreal Engine

Deep learning-based computer vision methods are transforming animal behavioural research. Transfer learning has enabled work in non-model species, but still requires hand-annotation of example footage, and is only performant in well-defined conditions. To help overcome these limitations, we developed replicAnt, a configurable pipeline implemented in Unreal Engine 5 and Python, designed to generate large and variable training datasets on consumer-grade hardware. replicAnt places 3D animal models into complex, procedurally generated environments, from which automatically annotated images can be exported. We demonstrate that synthetic data generated with replicAnt can significantly reduce the hand-annotation required to achieve benchmark performance in common applications such as animal detection, tracking, pose-estimation, and semantic segmentation. We also show that it increases the subject-specificity and domain-invariance of the trained networks, thereby conferring robustness. In some applications, replicAnt may even remove the need for hand-annotation altogether. It thus represents a significant step towards porting deep learning-based computer vision tools to the field.

Emerging biological insights enabled by high-resolution 3D motion data: promises, perspectives and pitfalls

Deconstructing motion to better understand it is a key prerequisite in the field of comparative biomechanics. Since Marey and Muybridge's work, technical constraints have been the largest limitation to motion capture and analysis, which, in turn, limited what kinds of questions biologists could ask or answer. Throughout the history of our field, conceptual leaps and significant technical advances have generally worked hand in hand. Recently, high-resolution, three-dimensional (3D) motion data have become easier to acquire, providing new opportunities for comparative biomechanics. We describe how adding a third dimension of information has fuelled major paradigm shifts, not only leading to a reinterpretation of long-standing scientific questions but also allowing new questions to be asked. In this paper, we highlight recent work published in Journal of Experimental Biology and influenced by these studies, demonstrating the biological breakthroughs made with 3D data. Although amazing opportunities emerge from these technical and conceptual advances, high-resolution data often come with a price. Here, we discuss challenges of 3D data, including low-throughput methodology, costly equipment, low sample sizes, and complex analyses and presentation. Therefore, we propose guidelines for how and when to pursue 3D high-resolution data. We also suggest research areas that are poised for major new biological advances through emerging 3D data collection.