Automated Planar Tracking the Waving Bodies of Multiple Zebrafish Swimming in Shallow Water

A review of 28 free animal-tracking software applications: current features and limitations

Well-quantified laboratory studies can provide a fundamental understanding of animal behavior in ecology, ethology and ecotoxicology research. These types of studies require observation and tracking of each animal in well-controlled and defined arenas, often for long timescales. Thus, these experiments produce long time series and a vast amount of data that require the use of software applications to automate the analysis and reduce manual annotation. In this review, we examine 28 free software applications for animal tracking to guide researchers in selecting the software that might best suit a particular experiment. We also review the algorithms in the tracking pipeline of the applications, explain how specific techniques can fit different experiments, and finally, expose each approach's weaknesses and strengths. Our in-depth review includes last update, type of platform, user-friendliness, off- or online video acquisition, calibration method, background subtraction and segmentation method, species, multiple arenas, multiple animals, identity preservation, manual identity correction, data analysis and extra features. We found, for example, that out of 28 programs, only 3 include a calibration algorithm to reduce image distortion and perspective problems that affect accuracy and can result in substantial errors when analyzing trajectories and extracting mobility or explored distance. In addition, only 4 programs can directly export in-depth tracking and analysis metrics, only 5 are suited for tracking multiple unmarked animals for more than a few seconds and only 11 have been updated in the period 2019-2021.

Parallel Fish School Tracking Based on Multiple Appearance Feature Detection

A parallel fish school tracking based on multiple-feature fish detection has been proposed in this paper to obtain accurate movement trajectories of a large number of zebrafish. Zebrafish are widely adapted in many fields as an excellent model organism. Due to the non-rigid body, similar appearance, rapid transition, and frequent occlusions, vision-based behavioral monitoring is still a challenge. A multiple appearance feature based fish detection scheme was developed by examining the fish head and center of the fish body based on shape index features. The proposed fish detection has the advantage of locating individual fishes from occlusions and estimating their motion states, which could ensure the stability of tracking multiple fishes. Moreover, a parallel tracking scheme was developed based on the SORT framework by fusing multiple features of individual fish and motion states. The proposed method was evaluated in seven video clips taken under different conditions. These videos contained various scales of fishes, different arena sizes, different frame rates, and various image resolutions. The maximal number of tracking targets reached 100 individuals. The correct tracking ratio was 98.60% to 99.86%, and the correct identification ratio ranged from 97.73% to 100%. The experimental results demonstrate that the proposed method is superior to advanced deep learning-based methods. Nevertheless, this method has real-time tracking ability, which can acquire online trajectory data without high-cost hardware configuration.

Zebrafish automatic monitoring system for conditioning and behavioral analysis

Studies using zebrafish (Danio rerio) in neuro-behavioural research are growing. Measuring fish behavior by computational methods is one of the most efficient ways to avoid human bias in experimental analyses, extending them to various approaches. Sometimes, thorough analyses are difficult to do, as fish can behave unpredictably during an experimental strategy. However, the analyses can be implemented in an automated way, using an online strategy and video processing for a complete assessment of the zebrafish behavior, based on the detection and tracking of fish during an activity. Here, a fully automatic conditioning and detailed analysis of zebrafish behavior is presented. Microcontrolled components were used to control the delivery of visual and sound stimuli, in addition to the concise amounts of food after conditioned stimuli for adult zebrafish groups in a conventional tank. The images were captured and processed for automatic detection of the fish, and the training of the fish was done in two evaluation strategies: simple and complex. In simple conditioning, the zebrafish showed significant responses from the second attempt, learning that the conditioned stimulus was a predictor of food presentation in a specific space of the tank, where the food was dumped. When the fish were subjected to two stimuli for decision-making in the food reward, the zebrafish obtained better responses to red light stimuli in relation to vibration. The behavior change was clear in stimulated fish in relation to the control group, thus, the distances traveled and the speed were greater, while the polarization was lower in stimulated fish. This automated system allows for the conditioning and assessment of zebrafish behavior online, with greater stability in experiments, and in the analysis of the behavior of individual fish or fish schools, including learning and memory studies.

Zebrafish tracking using YOLOv2 and Kalman filter

Fish show rapid movements in various behavioral activities or associated with the presence of food. However, in periods of rapid movement, the rate at which occlusion occurs among the fish is quite high, causing inconsistency in the detection and tracking of fish, hindering the fish's identity and behavioral trajectory over a long period of time. Although some algorithms have been proposed to solve these problems, most of their applications were made in groups of fish that swim in shallow water and calm behavior, with few sudden movements. To solve these problems, a convolutional network of object recognition, YOLOv2, was used to delimit the region of the fish heads to optimize individual fish detection. In the tracking phase, the Kalman filter was used to estimate the best state of the fish's head position in each frame and, subsequently, the trajectories of each fish were connected among the frames. The results of the algorithm show adequate performances in the trajectories of groups of zebrafish that exhibited rapid movements.

epiTracker: A Framework for Highly Reliable Particle Tracking for the Quantitative Analysis of Fish Movements in Tanks

Behavioral analysis of moving animals relies on a faithful recording and track analysis to extract relevant parameters of movement. To study group behavior and social interactions, often simultaneous analyses of individuals are required. To detect social interactions, for example to identify the leader of a group as opposed to followers, one needs an error-free segmentation of individual tracks throughout time. While automated tracking algorithms exist that are quick and easy to use, inevitable errors will occur during tracking. To solve this problem, we introduce a robust algorithm called epiTracker for segmentation and tracking of multiple animals in two-dimensional (2D) videos along with an easy-to-use correction method that allows one to obtain error-free segmentation. We have implemented two graphical user interfaces to allow user-friendly control of the functions. Using six labeled 2D datasets, the effort to obtain accurate labels is quantified and compared to alternative available software solutions. Both the labeled datasets and the software are publicly available.

The zebrafish subcortical social brain as a model for studying social behavior disorders

Social behaviors are essential for the survival and reproduction of social species. Many, if not most, neuropsychiatric disorders in humans are either associated with underlying social deficits or are accompanied by social dysfunctions. Traditionally, rodent models have been used to model these behavioral impairments. However, rodent assays are often difficult to scale up and adapt to high-throughput formats, which severely limits their use for systems-level science. In recent years, an increasing number of studies have used zebrafish (Danio rerio) as a model system to study social behavior. These studies have demonstrated clear potential in overcoming some of the limitations of rodent models. In this Review, we explore the evolutionary conservation of a subcortical social brain between teleosts and mammals as the biological basis for using zebrafish to model human social behavior disorders, while summarizing relevant experimental tools and assays. We then discuss the recent advances gleaned from zebrafish social behavior assays, the applications of these assays to studying related disorders, and the opportunities and challenges that lie ahead.

Methods for the effective study of collective behavior in a radial arm maze

Collective behaviors are observed throughout nature, from bacterial colonies to human societies. Important theoretical breakthroughs have recently been made in understanding why animals produce group behaviors and how they coordinate their activities, build collective structures, and make decisions. However, standardized experimental methods to test these findings have been lacking. Notably, easily and unambiguously determining the membership of a group and the responses of an individual within that group is still a challenge. The radial arm maze is presented here as a new standardized method to investigate collective exploration and decision-making in animal groups. This paradigm gives individuals within animal groups the opportunity to make choices among a set of discrete alternatives, and these choices can easily be tracked over long periods of time. We demonstrate the usefulness of this paradigm by performing a set of refuge-site selection experiments with groups of fish. Using an open-source, robust custom image-processing algorithm, we automatically counted the number of animals in each arm of the maze to identify the majority choice. We also propose a new index to quantify the degree of group cohesion in this context. The radial arm maze paradigm provides an easy way to categorize and quantify the choices made by animals. It makes it possible to readily apply the traditional uses of the radial arm maze with single animals to the study of animal groups. Moreover, it opens up the possibility of studying questions specifically related to collective behaviors.

Aquatic Toxic Analysis by Monitoring Fish Behavior Using Computer Vision: A Recent Progress

Video tracking based biological early warning system achieved a great progress with advanced computer vision and machine learning methods. Ability of video tracking of multiple biological organisms has been largely improved in recent years. Video based behavioral monitoring has become a common tool for acquiring quantified behavioral data for aquatic risk assessment. Investigation of behavioral responses under chemical and environmental stress has been boosted by rapidly developed machine learning and artificial intelligence. In this paper, we introduce the fundamental of video tracking and present the pioneer works in precise tracking of a group of individuals in 2D and 3D space. Technical and practical issues suffered in video tracking are explained. Subsequently, the toxic analysis based on fish behavioral data is summarized. Frequently used computational methods and machine learning are explained with their applications in aquatic toxicity detection and abnormal pattern analysis. Finally, advantages of recent developed deep learning approach in toxic prediction are presented.

Tracking Nemo: Help Scientists Understand Zebrafish Behavior

The advent of automated tracking software has significantly reduced the time required to record movement trajectories, thereby facilitating behavioral studies of zebrafish. However, results are substantially influenced by tracking errors, such as loss and misidentification of individuals. In this study, we present the development of an online citizen science platform, Tracking Nemo, to improve data accuracy on swimming trajectories of zebrafish groups. As an online extension of software for tracking the position of zebrafish from video recordings, Tracking Nemo offers volunteers the opportunity to contribute to science by manually correcting tracked trajectory data from their personal computers. Researchers can upload their videos that require human intervention for correcting and validating the data. Citizen scientists can monitor their contributions through a leaderboard system, which is designed to strengthen participant retention and contribution by tapping into intrinsic and extrinsic motivations. Tracking Nemo is expected to help scientists improve data accuracy through the involvement of citizen scientists, who, in turn, engage in an authentic research activity and learn more about the behavior of zebrafish.

ToxId: an efficient algorithm to solve occlusions when tracking multiple animals

Video analysis of animal behaviour is widely used in fields such as ecology, ecotoxicology, and evolutionary research. However, when tracking multiple animals, occlusion and crossing are problematic, especially when the identity of each individual needs to be preserved. We present a new algorithm, ToxId, which preserves the identity of multiple animals by linking trajectory segments using their intensity histogram and Hu-moments. We verify the performance and accuracy of our algorithm using video sequences with different animals and experimental conditions. The results show that our algorithm achieves state-of-the-art accuracy using an efficient approach without the need of learning processes, complex feature maps or knowledge of the animal shape. ToxId is also computationally efficient, has low memory requirements, and operates without accessing future or past frames.

Feature point based 3D tracking of multiple fish from multi-view images

A feature point based method is proposed for tracking multiple fish in 3D space. First, a simplified representation of the object is realized through construction of two feature point models based on its appearance characteristics. After feature points are classified into occluded and non-occluded types, matching and association are performed, respectively. Finally, the object's motion trajectory in 3D space is obtained through integrating multi-view tracking results. Experimental results show that the proposed method can simultaneously track 3D motion trajectories for up to 10 fish accurately and robustly.