Cow key point detection in indoor housing conditions with a deep learning model

Lameness in dairy cattle is a costly and highly prevalent problem that affects all aspects of sustainable dairy production, including animal welfare. Automation of gait assessment would allow monitoring of locomotion in which the cows' walking patterns can be evaluated frequently and with limited labor. With the right interpretation algorithms, this could result in more timely detection of locomotion problems. This in turn would facilitate timely intervention and early treatment, which is crucial to reduce the effect of abnormal behavior and pain on animal welfare. Gait features of dairy cows can potentially be derived from key points that locate crucial anatomical points on a cow's body. The aim of this study is 2-fold: (1) to demonstrate automation of the detection of dairy cows' key points in a practical indoor setting with natural occlusions from gates and races, and (2) to propose the necessary steps to postprocess these key points to make them suitable for subsequent gait feature calculations. Both the automated detection of key points as well as the postprocessing of them are crucial prerequisites for camera-based automated locomotion monitoring in a real farm environment. Side-view video footage of 34 Holstein-Friesian dairy cows, captured when exiting the milking parlor, were used for model development. From these videos, 758 samples of 2 successive frames were extracted. A previously developed deep learning model called T-LEAP was trained to detect 17 key points on cows in our indoor farm environment with natural occlusions. To this end, the dataset of 758 samples was randomly split into a train (n = 22 cows; no. of samples = 388), validation (n = 7 cows; no. of samples = 108), and test dataset (n = 15 cows; no. of samples = 262). The performance of T-LEAP to automatically assign key points in our indoor situation was assessed using the average percentage of correctly detected key points using a threshold of 0.2 of the head length (PCKh0.2). The model's performance on the test set achieved a good result with PCKh0.2: 89% on all 17 key points together. Detecting key points on the back (n = 3 key points) of the cow had the poorest performance PCKh0.2: 59%. In addition to the indoor performance of the model, a more detailed study of the detection performance was conducted to formulate postprocessing steps necessary to use these key points for gait feature calculations and subsequent automated locomotion monitoring. This detailed study included the evaluation of the detection performance in multiple directions. This study revealed that the performance of the key points on a cows' back were the poorest in the horizontal direction. Based on this more in-depth study, we recommend the implementation of the outlined postprocessing techniques to address the following issues: (1) correcting camera distortion, (2) rectifying erroneous key point detection, and (3) establishing the necessary procedures for translating hoof key points into gait features.

Invited review: integration of technologies and systems for precision animal agriculture-a case study on precision dairy farming

Precision livestock farming (PLF) offers a strategic solution to enhance the management capacity of large animal groups, while simultaneously improving profitability, efficiency, and minimizing environmental impacts associated with livestock production systems. Additionally, PLF contributes to optimizing the ability to manage and monitor animal welfare while providing solutions to global grand challenges posed by the growing demand for animal products and ensuring global food security. By enabling a return to the "per animal" approach by harnessing technological advancements, PLF enables cost-effective, individualized care for animals through enhanced monitoring and control capabilities within complex farming systems. Meeting the nutritional requirements of a global population exponentially approaching ten billion people will likely require the density of animal proteins for decades to come. The development and application of digital technologies are critical to facilitate the responsible and sustainable intensification of livestock production over the next several decades to maximize the potential benefits of PLF. Real-time continuous monitoring of each animal is expected to enable more precise and accurate tracking and management of health and well-being. Importantly, the digitalization of agriculture is expected to provide collateral benefits of ensuring auditability in value chains while assuaging concerns associated with labor shortages. Despite notable advances in PLF technology adoption, a number of critical concerns currently limit the viability of these state-of-the-art technologies. The potential benefits of PLF for livestock management systems which are enabled by autonomous continuous monitoring and environmental control can be rapidly enhanced through an Internet of Things approach to monitoring and (where appropriate) closed-loop management. In this paper, we analyze the multilayered network of sensors, actuators, communication, networking, and analytics currently used in PLF, focusing on dairy farming as an illustrative example. We explore the current state-of-the-art, identify key shortcomings, and propose potential solutions to bridge the gap between technology and animal agriculture. Additionally, we examine the potential implications of advancements in communication, robotics, and artificial intelligence on the health, security, and welfare of animals.

Intelligent Perception-Based Cattle Lameness Detection and Behaviour Recognition: A Review

Simple Summary

Cattle lameness detection as well as behaviour recognition are the two main objectives in the applications of precision livestock farming (PLF). Over the last five years, the development of smart sensors, big data, and artificial intelligence has offered more automatic tools. In this review, we discuss over 100 papers that used automated techniques to detect cattle lameness and to recognise animal behaviours. To assist researchers and policy-makers in promoting various livestock technologies for monitoring cattle welfare and productivity, we conducted a comprehensive investigation of intelligent perception for cattle lameness detection and behaviour analysis in the PLF domain. Based on the literature review, we anticipate that PLF will develop in an objective, autonomous, and real-time direction. Additionally, we suggest that further research should be dedicated to improving the data quality, modeling accuracy, and commercial availability.

Abstract

The growing world population has increased the demand for animal-sourced protein. However, animal farming productivity is faced with challenges from traditional farming practices, socioeconomic status, and climate change. In recent years, smart sensors, big data, and deep learning have been applied to animal welfare measurement and livestock farming applications, including behaviour recognition and health monitoring. In order to facilitate research in this area, this review summarises and analyses some main techniques used in smart livestock farming, focusing on those related to cattle lameness detection and behaviour recognition. In this study, more than 100 relevant papers on cattle lameness detection and behaviour recognition have been evaluated and discussed. Based on a review and a comparison of recent technologies and methods, we anticipate that intelligent perception for cattle behaviour and welfare monitoring will develop towards standardisation, a larger scale, and intelligence, combined with Internet of things (IoT) and deep learning technologies. In addition, the key challenges and opportunities of future research are also highlighted and discussed.

The Importance of Low Daily Risk for the Prediction of Treatment Events of Individual Dairy Cows with Sensor Systems

The prediction of health disorders is the goal of many sensor systems in dairy farming. Although mastitis and lameness are the most common health disorders in dairy cows, these diseases or treatments are a rare event related to a single day and cow. A number of studies already developed and evaluated models for classifying cows in need of treatment for mastitis and lameness with machine learning methods, but few have illustrated the effects of the positive predictive value (PPV) on practical application. The objective of this study was to investigate the importance of low-frequency treatments of mastitis or lameness for the applicability of these classification models in practice. Data from three German dairy farms contained animal individual sensor data (milkings, activity, feed intake) and were classified using machine learning models developed in a previous study. Subsequently, different risk criteria (previous treatments, information from milk recording, early lactation) were designed to isolate high-risk groups. Restricting selection to cows with previous mastitis or hoof treatment achieved the highest increase in PPV from 0.07 to 0.20 and 0.15, respectively. However, the known low daily risk of a treatment per cow remains the critical factor that prevents the reduction of daily false-positive alarms to a satisfactory level. Sensor systems should be seen as additional decision-support aid to the farmers’ expert knowledge.

Prediction of lameness using automatically recorded activity, behavior and production data in post-parturient Irish dairy cows

BACKGROUND

Although visual locomotion scoring is inexpensive and simplistic, it is also time consuming and subjective. Automated lameness detection methods have been developed to replace the visual locomotion scoring and aid in early and accurate detection. Several types of sensors are measuring traits such as activity, lying behavior or temperature. Previous studies on automatic lameness detection have been unable to achieve high accuracy in combination with practical implementation in a on farm commercial setting. The objective of our research was to develop a prediction model for lameness in dairy cattle using a combination of remote sensor technology and other animal records that will translate sensor data into easy to interpret classified locomotion information for the farmer. During an 11-month period, data from 164 Holstein-Friesian dairy cows were gathered, housed at an Irish research farm. A neck-mounted accelerometer was used to gather behavioral metrics, additional automatically recorded data consisted of milk production and live weight. Locomotion scoring data were manually recorded, using a one-to-five scale (1 = non-lame, 5 = severely lame). Locomotion scores where then used to label the cows as sound (locomotion score 1) or unsound (locomotion score ≥ 2). Four supervised classification models, using a gradient boosted decision tree machine learning algorithm, were constructed to investigate whether cows could be classified as sound or unsound. Data available for model building included behavioral metrics, milk production and animal characteristics.

RESULTS

The resulting models were constructed using various combinations of the data sources. The accuracy of the models was then compared using confusion matrices, receiver-operator characteristic curves and calibration plots. The model which achieved the highest performance according to the accuracy measures, was the model combining all the available data, resulting in an area under the curve of 85% and a sensitivity and specificity of 78%.

CONCLUSION

These results show that 85% of this model's predictions were correct in identifying cows as sound or unsound, showing that the use of a neck-mounted accelerometer, in combination with production and other animal data, has potential to replace visual locomotion scoring as lameness detection method in dairy cows.

Using Sensor Data to Detect Lameness and Mastitis Treatment Events in Dairy Cows: A Comparison of Classification Models

The aim of this study was to develop classification models for mastitis and lameness treatments in Holstein dairy cows as the target variables based on continuous data from herd management software with modern machine learning methods. Data was collected over a period of 40 months from a total of 167 different cows with daily individual sensor information containing milking parameters, pedometer activity, feed and water intake, and body weight (in the form of differently aggregated data) as well as the entered treatment data. To identify the most important predictors for mastitis and lameness treatments, respectively, Random Forest feature importance, Pearson’s correlation and sequential forward feature selection were applied. With the selected predictors, various machine learning models such as Logistic Regression (LR), Support Vector Machine (SVM), K-nearest neighbors (KNN), Gaussian Naïve Bayes (GNB), Extra Trees Classifier (ET) and different ensemble methods such as Random Forest (RF) were trained. Their performance was compared using the receiver operator characteristic (ROC) area-under-curve (AUC), as well as sensitivity, block sensitivity and specificity. In addition, sampling methods were compared: Over- and undersampling as compensation for the expected unbalanced training data had a high impact on the ratio of sensitivity and specificity in the classification of the test data, but with regard to AUC, random oversampling and SMOTE (Synthetic Minority Over-sampling) even showed significantly lower values than with non-sampled data. The best model, ET, obtained a mean AUC of 0.79 for mastitis and 0.71 for lameness, respectively, based on testing data from practical conditions and is recommended by us for this type of data, but GNB, LR and RF were only marginally worse, and random oversampling and SMOTE even showed significantly lower values than without sampling. We recommend the use of these models as a benchmark for similar self-learning classification tasks. The classification models presented here retain their interpretability with the ability to present feature importances to the farmer in contrast to the “black box” models of Deep Learning methods.

Grazing Cow Behavior's Association with Mild and Moderate Lameness

Accelerometer-based mobility scoring has focused on cow behaviors such as lying and walking. Accuracy levels as high as 91% have been previously reported. However, there has been limited replication of results. Here, measures previously identified as indicative of mobility, such as lying bouts and walking time, were examined. On a research farm and a commercial farm, 63 grazing cows' behavior was monitored in four trials (16, 16, 16, and 15 cows) using leg-worn accelerometers. Seventeen good mobility (score 0), 23 imperfect mobility (score 1), and 22 mildly impaired mobility (score 2) cows were monitored. Only modest associations with activity, standing, and lying events were found. Thus, behavior monitoring appears to be insufficient to discern mildly and moderately impaired mobility of grazing cows.

Boosted trees to predict pneumonia, growth, and meat percentage of growing-finishing pigs1

In pig production, efficiency is benefiting from uniform growth in pens resulting in single deliveries from a pen of possibly all animals in the targeted weight range. Abnormalities, like pneumonia or aberrant growth, reduce production efficiency as it reduces the uniformity and might cause multiple deliveries per batch and pigs delivered with a low meat yield or outside the targeted weight range. Early identification of pigs prone to develop these abnormalities, for example, at the onset of the growing-finishing phase, would help to prevent heterogeneous pens through management interventions. Data about previous production cycles at the farm combined with data from the piglet’s own history may help in identifying these abnormalities. The aim of this study, therefore, was to predict at the onset of the growing-finishing phase, that is, at 3 mo in advance, deviant pigs at slaughter with a machine-learning technique called boosted trees. The dataset used was extracted from the farm management system of a research center. It contained over 70,000 records of individual pigs born between 2004 and 2016, including information on, for example, offspring, litter size, transfer dates between production stages, their respective locations within the barns, and individual live-weights at several production stages. Results obtained on an independent test set showed that at a 90% specificity rate, the sensitivity was 16% for low meat percentage, 20% for pneumonia and 36% for low lifetime growth rate. For low lifetime growth rate, this meant an almost three times increase in positive predictive value compared to the current situation. From these results, it was concluded that routine performance information available at the onset of the growing-finishing phase combined with data about previous production cycles formed a moderate base to identify pigs prone to develop pneumonia (AUC > 0.60) and a good base to identify pigs prone to develop growth aberrations (AUC > 0.70) during the growing-finishing phase. The mentioned information, however, was not a sufficient base to identify pigs prone to develop low meat percentage (AUC < 0.60). The shown ability to identify growth aberrations and pneumonia can be considered a good first step towards the development of an early warning system for pigs in the growing-finishing phase.

Automatic lameness detection in cattle

There is an increasing demand for health and welfare monitoring in modern dairy farming. The development of various innovative techniques aims at improving animal behaviour monitoring and thus animal welfare indicators on-farm. Automated lameness detection systems have to be valid, reliable and practicable to be applied in veterinary practice or under farm conditions. The objective of this literature review was to describe the current automated systems for detection of lameness in cattle, which have been recently developed and investigated for application in dairy research and practice. The automatic methods of lameness detection broadly fall into three categories: kinematic, kinetic and indirect methods. The performance of the methods were compared with the reference standard (locomotion score and/or lesion score) and evaluated based on level-based scheme defining the degree of development (level I, sensor technique; level II, validation of algorithm; level III, performance for detection of lameness and/or lesion; level IV, decision support with early warning system). Many scientific studies have been performed on levels I-III, but there are no studies of level IV technology. The adoption rate of automated lameness detection systems by herd managers mainly yields returns on investment by the early identification of lame cows. Long-term studies, using validated automated lameness detection systems aiming at early lameness detection, are still needed in order to improve welfare and production under field conditions.

Animal board invited review: precision livestock farming for dairy cows with a focus on oestrus detection

Dairy cows are high value farm animals requiring careful management to achieve the best results. Since the advent of robotic and high throughput milking, the traditional few minutes available for individual human attention daily has disappeared and new automated technologies have been applied to improve monitoring of dairy cow production, nutrition, fertility, health and welfare. Cows milked by robots must meet legal requirements to detect healthy milk. This review focuses on emerging technical approaches in those areas of high cost to the farmer (fertility, metabolic disorders, mastitis, lameness and calving). The availability of low cost tri-axial accelerometers and wireless telemetry has allowed accurate models of behaviour to be developed and sometimes combined with rumination activity detected by acoustic sensors to detect oestrus; other measures (milk and skin temperature, electronic noses, milk yield) have been abandoned. In-line biosensors have been developed to detect markers for ovulation, pregnancy, lactose, mastitis and metabolic changes. Wireless telemetry has been applied to develop boluses for monitoring the rumen pH and temperature to detect metabolic disorders. Udder health requires a multisensing approach due to the varying inflammatory responses collectively described as mastitis. Lameness can be detected by walk over weigh cells, but also by various types of video image analysis and speed measurement. Prediction and detection of calving time is an area of active research mostly focused on behavioural change.

Lameness Detection in Dairy Cows: Part 2. Use of Sensors to Automatically Register Changes in Locomotion or Behavior

Simple Summary

As lame cows produce less milk and tendto have other health problems, finding and treating lame cows is very importantfor farmers. Sensors that measure behaviors associated with lameness in cowscan help by alerting the farmer of those cows in need of treatment. This reviewgives an overview of sensors for automated lameness detection and discussessome practical considerations for investigating and applying such systems inpractice.

Abstract

Despite the research on opportunities toautomatically measure lameness in cattle, lameness detection systems are notwidely available commercially and are only used on a few dairy farms. However, farmers need to be aware of the lame cows in their herds in order treat themproperly and in a timely fashion. Many papers have focused on the automatedmeasurement of gait or behavioral cow characteristics related to lameness. Inorder for such automated measurements to be used in a detection system, algorithms to distinguish between non-lame and mildly or severely lame cowsneed to be developed and validated. Few studies have reached this latter stageof the development process. Also, comparison between the different approachesis impeded by the wide range of practical settings used to measure the gait or behavioralcharacteristic (e.g., measurements during normal farming routine or duringexperiments; cows guided or walking at their own speed) and by the differentdefinitions of lame cows. In the majority of the publications, mildly lame cowsare included in the non-lame cow group, which limits the possibility of alsodetecting early lameness cases. In this review, studies that used sensortechnology to measure changes in gait or behavior of cows related to lamenessare discussed together with practical considerations when conducting lamenessresearch. In addition, other prerequisites for any lameness detection system onfarms (e.g., need for early detection, real-time measurements) are discussed.

Lameness detection in dairy cattle: single predictor v. multivariate analysis of image-based posture processing and behaviour and performance sensing

The objective of this study was to evaluate if a multi-sensor system (milk, activity, body posture) was a better classifier for lameness than the single-sensor-based detection models. Between September 2013 and August 2014, 3629 cow observations were collected on a commercial dairy farm in Belgium. Human locomotion scoring was used as reference for the model development and evaluation. Cow behaviour and performance was measured with existing sensors that were already present at the farm. A prototype of three-dimensional-based video recording system was used to quantify automatically the back posture of a cow. For the single predictor comparisons, a receiver operating characteristics curve was made. For the multivariate detection models, logistic regression and generalized linear mixed models (GLMM) were developed. The best lameness classification model was obtained by the multi-sensor analysis (area under the receiver operating characteristics curve (AUC)=0.757±0.029), containing a combination of milk and milking variables, activity and gait and posture variables from videos. Second, the multivariate video-based system (AUC=0.732±0.011) performed better than the multivariate milk sensors (AUC=0.604±0.026) and the multivariate behaviour sensors (AUC=0.633±0.018). The video-based system performed better than the combined behaviour and performance-based detection model (AUC=0.669±0.028), indicating that it is worthwhile to consider a video-based lameness detection system, regardless the presence of other existing sensors in the farm. The results suggest that Θ2, the feature variable for the back curvature around the hip joints, with an AUC of 0.719 is the best single predictor variable for lameness detection based on locomotion scoring. In general, this study showed that the video-based back posture monitoring system is outperforming the behaviour and performance sensing techniques for locomotion scoring-based lameness detection. A GLMM with seven specific variables (walking speed, back posture measurement, daytime activity, milk yield, lactation stage, milk peak flow rate and milk peak conductivity) is the best combination of variables for lameness classification. The accuracy on four-level lameness classification was 60.3%. The accuracy improved to 79.8% for binary lameness classification. The binary GLMM obtained a sensitivity of 68.5% and a specificity of 87.6%, which both exceed the sensitivity (52.1%±4.7%) and specificity (83.2%±2.3%) of the multi-sensor logistic regression model. This shows that the repeated measures analysis in the GLMM, taking into account the individual history of the animal, outperforms the classification when thresholds based on herd level (a statistical population) are used.