SpringerLink
Log in

Advanced dairy cow monitoring: enhanced detection with precision 3D tracking

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Ensuring the welfare of dairy cows requires precise monitoring of their daily exercise to evaluate their physical health. This necessitates innovative methods beyond traditional motion sensors. We present a novel method that integrates an enhanced YOLOv5s object detection model with the DeepSORT multi-object tracking algorithm to meticulously track dairy cow movements, providing holistic information about their health. Our research started with the establishment of a dedicated dataset tailored for cow detection. We then segmented the detection scope to focus on specific regions of interest. Within the modified YOLOv5s model, we replaced the standard CSPDarknet53 backbone with DenseNet to achieve deep separable convolution and feature reorganization modules, leading to reduced parameters, augmented feature expression, and better information flow. In particular, the SPD-Conv module was incorporated to retain intricate details, essential for detecting smaller and low-resolution targets. The transition from Generalized Intersection over Union (GIoU) Loss to Complete Intersection over Union (CIoU) loss improved detection accuracy and sped up model convergence. Our clustering approach, based on the elbow rule, optimized K-means clustering, enhancing speed and precision. For multi-object tracking, the DeepSORT model was tailored to cater to varying cow sizes, and we chose an algorithm to associate appearance information. We converted pixel data into real-world distances, providing exact 3D cow movement metrics. Experimental validation confirmed the efficacy of our approach. Our enhanced model surpassed the original YOLOv5s in performance by 11.1% for accuracy (97.4%), 9.6% for recall (97.8%), and 11.0% for average accuracy (98.2%). The comprehensive accuracy stood at 92.1% for our model. In conclusion, our innovative methodology offers a non-invasive means to monitor dairy cow exercise, paving the way for advanced health assessment techniques in the dairy sector.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Data Availability

Raw data were generated at Bingshen Cows Cooperative Ranch. Derived data supporting the findings of this study are available from the corresponding author on request.

References

  1. Borchers MR, Chang YM, Tsai IC, Wadsworth BA, Bewley JM (2016) A validation of technologies monitoring dairy cow feeding, ruminating, and lying behaviors. J Dairy Sci 99(9):7458–7468

    Article  Google Scholar 

  2. Zin TT, Phyo CN, Tin P, Hama H, Kobayashi I (2018) Image technology based cow identification system using deep learning. In: Proceedings of the international multiconference of engineers and computer scientists, pp 1–6. https://www.iaeng.org/publication/IMECS2018/IMECS2018_pp1-6.pdf

  3. Peng Y, Kondo N, Fujiura T, Suzuki T, Wulandari Yoshioka H, Itoyama E (2019) Classification of multiple cattle behavior patterns using a recurrent neural network with long short-term memory and inertial measurement units. Comput Electron Agric 157:136–143

    Article  Google Scholar 

  4. Shen W, Hu H, Dai B, Wei X, Sun J, Jiang L, Sun Y (2019) Individual identification of dairy cows based on convolutional neural networks. Multimed Tools Appl 78(11):14863–14880

    Google Scholar 

  5. Qiao Y, Su D, Kong H, Sukkarieh S, Lomax S, Clark C (2019) Individual cattle identification using a deep learning based framework. Comput Electron Agric 157:136–143

    Google Scholar 

  6. Qiao Y, Su D, Kong H, Sukkarieh S, Lomax S (2020) Bilstm-based individual cattle identification for automated precision livestock farming. In: IEEE international conference on robotics and automation (ICRA). https://ieeexplore.ieee.org/document/9196472

  7. Qiao Y, Clark C, Lomax S, Kong H, Su D, Sukkarieh S (2021) Automated individual cattle identification using video data: A unified deep learning architecture approach. Frontiers in Animal Science 2

  8. Shakeel PM, Bb Mohd Aboobaider (2022) A deep learning-based cow behavior recognition scheme for improving cattle behavior modeling in smart farming. Internet of Things 16:100539

    Article  Google Scholar 

  9. Hossain ME, Kabir MA, Zheng L, Swain DL, McGrath S, Medway J (2022) A systematic review of machine learning techniques for cattle identification: Datasets, methods and future directions. Artificial Intelligence in Agriculture

  10. Noe SM, Zin TT, Tin P, Kobayashi I (2023) Comparing state-of-the-art deep learning algorithms for the automated detection and tracking of black cattle. Sensors 23(1):532

    Article  Google Scholar 

  11. Barney S, Dlay S, Crowe A, Kyriazakis I, Leach M (2023) Deep learning pose estimation for multi-cattle lameness detection. Sci Rep 13(1):24067

    Article  Google Scholar 

  12. Gao G, Wang C, Wang J, Lv Y, Li Q, Ma Y, Zhang X, Li Z, Chen G (2023) Cnn-bi-lstm: A complex environment-oriented cattle behavior classification network based on the fusion of cnn and bi-lstm. Sensors 23(18):7714

    Article  Google Scholar 

  13. Hua Z, Wang Z, Xu X, Kong X, Song H (2023) An effective posec3d model for typical action recognition of dairy cows based on skeleton features. Comput Electron Agric 212:108152. https://doi.org/10.1016/j.compag.2023.108152

    Article  Google Scholar 

  14. Li R, Wen Y, Zhang S, Xu X, Ma B, Song H (2024) Automated measurement of beef cattle body size via key point detection and monocular depth estimation. Expert Syst Appl 244:123042

    Article  Google Scholar 

  15. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778

  16. Sunkara R, Luo T (2022) No more strided convolutions or pooling: A new cnn building block for low-resolution images and small objects. ar**v:2208.03641

  17. Hastie T, Tibshirani R, Friedman J (2009) The Elements of Statistical Learning. Springer, ???. https://www.springer.com/gp/book/9780387848570

  18. Chen M, Girshick R, He K, Dollar P (2020) Enhancing geometric factors in model learning and inference for object detection and instance segmentation. ar**v:2006.05430

  19. Zheng K, Kong J, Tian L, Li B, Li H, Zhou J (2021) Distance-iou loss: Faster and better learning for bounding box regression. ar**v:2103.07737

  20. Bewley A, Zongyuan G, Ramos F, Upcroft B (2016) Simple online and realtime tracking with a deep association metric. In: International conference on image processing (ICIP), pp 3464–3468. https://ieeexplore.ieee.org/document/7533053

  21. Zheng K, Kong J, Tian L, Li B, Li H, Zhou J (2023) Hand-over-face occlusion and distance adaptive heart rate detection based on imaging photoplethysmography and pixel distance in online learning. Biomedical Signal Processing and Control 85

  22. Burnett TA, Madureira AML, Silper BF, al (2017) Integrating an automated activity monitor into an artificial insemination program and the associated risk factors affecting reproductive performance of dairy cows. J Dairy Sci 100(6):5005–5018

  23. Thorndike RL (1953) Who belongs in the family? Psychometrika 18(4):267–276

    Article  Google Scholar 

  24. Tibshirani R, Walther G, Hastie T (2001) Estimating the number of clusters in a data set via the gap statistic. J R Stat Soc Ser B Stat Methodol 63(2):411–423

    Article  MathSciNet  Google Scholar 

  25. Rousseeuw PJ (1987) Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65

    Article  Google Scholar 

  26. Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S (2019) Generalized intersection over union: A metric and a loss for bounding box regression. ar**v:1902.09630

  27. Tian F, Wang R, Liu M, Wang Z, Li F, Wang Z (2013) Oestrus detection and prediction in dairy cows based on neural networks. Trans Chin Soc Agric Mach 44(S1):277–281

    Google Scholar 

  28. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on computer vision and pattern recognition (CVPR), pp 4510–4520

  29. Zhou K, Yang Y, Cavallaro A, **ang T (2019) Omni-scale feature learning for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1406–1414

  30. Zhou K, Yang Y, Cavallaro A, **ang T (2021) Learning generalisable omni-scale representations for person re-identification. IEEE Transactions on Pattern Analysis and Machine Intelligence

  31. Zhu X, Wang Y, Dai J, Yuan L, Wei Y (2017) Flow-guided feature aggregation for video object detection. In: Proceedings of the IEEE international conference on computer vision, pp 408–417

  32. Sun P, Zhang R, Jiang Y, Kong T, Xu C, Zhan W, Tomizuka M, Li L, Yuan Z, Wang C,et al (2021) Sparse r-cnn: End-to-end object detection with learnable proposals. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14454–14463

  33. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. ar**v:1804.02767

  34. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: Single shot multibox detector. In: Computer vision–ECCV 2016: 14th european conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, pp 21–37. Springer

  35. Ju M, Luo J, Zhang P, He M, Luo H (2019) A simple and efficient network for small target detection. IEEE Access 7:85771–85781

    Article  Google Scholar 

  36. Cui Y, Yang L, Liu D (2022) Dynamic proposals for efficient object detection. ar**v:2207.05252

  37. Cui Y (2022) Dynamic feature aggregation for efficient video object detection. In: Proceedings of the asian conference on computer vision, pp 944–960

  38. Shi W, Bao S, Tan D (2019) Ffessd: An accurate and efficient single-shot detector for target detection. Appl Sci 9(20):4276

    Article  Google Scholar 

  39. Zhang T, Guo H, Jiao Q, Zhang Q, Han J (2023) Efficient rgb-t tracking via cross-modality distillation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5404–5413

  40. Zhang T, Liu X, Zhang Q, Han J (2021) Siamcda: Complementarity-and distractor-aware rgb-t tracking based on siamese network. IEEE Trans Circ Syst Video Tech 32(3):1403–1417

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Key Technologies Research and Development Program of China, Subproject: ‘Development of an Intelligent Inspection Robot for Health Assessment in Beef Cattle Factory Farming’ (Project No. 2023YFD2000704-2)

Author information

Authors and Affiliations

  1. College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai’an, 271018, Shandong, China

    Ranran Wang, Yingxiu Li & Fuyang Tian

  2. Department of Computer Science, Faculty of Engineering, University of Sheffield, Sheffield, S1 4DP, UK

    Peng Yue

  3. Faculty of Agriculture, Iwate University, Morioka, 020-8550, Iwate, Japan

    Chunhong Yuan

  4. Faculty of Science and Engineering, Iwate University, Morioka, 020-8550, Iwate, Japan

    **n Lu

Authors
  1. Ranran Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yingxiu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peng Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chunhong Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fuyang Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. **n Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Fuyang Tian or **n Lu.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, R., Li, Y., Yue, P. et al. Advanced dairy cow monitoring: enhanced detection with precision 3D tracking. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-19791-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-19791-8

Keywords

Search

Navigation