• Chinese Core Journal
  • Chinese Science Citation Database (CSCD) Source journal
  • Journal of Citation Report of Chinese S&T Journals (Core Edition)
Advanced Search
SONG Huaibo, WANG Yunfei, DENG Hongxing, et al. Research progress of intelligent monitoring technology for large-scale dairy cows based on video analysis[J]. Journal of South China Agricultural University, 2024, 45(5): 649-660. DOI: 10.7671/j.issn.1001-411X.202403038
Citation: SONG Huaibo, WANG Yunfei, DENG Hongxing, et al. Research progress of intelligent monitoring technology for large-scale dairy cows based on video analysis[J]. Journal of South China Agricultural University, 2024, 45(5): 649-660. DOI: 10.7671/j.issn.1001-411X.202403038
shu

Research progress of intelligent monitoring technology for large-scale dairy cows based on video analysis

  • Key Laboratory of Agricultural Internet of Things, Ministry of Agriculture and Rural Affairs/College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, China

More Information
  • Author Bio:

    SONG Huaibo: 　　宋怀波，博士，教授，博士生导师。农业农村部农业物联网重点实验室副主任，“科创中国”农业工程科技服务团高级专家、农业工程学会高级会员、陕西省图形图像学会理事，中国农业工程学会畜牧工程分会第十届理事会理事，中国农学会农业监测预警分会理事、农业工程学会青年工作委员会委员，人工智能学会智能农业专委会会员，中国畜牧兽医学会信息技术分会会员、陕西省计算机学会会员。主要从事大型畜禽健康行为智能识别技术、农情信息获取及识别等方面的理论研究。先后主持国家自然科学基金青年项目，国家重点研发项目子课题，陕西省重点研发计划项目，陕西省自然科学基金，中央高校基本科研业务经费，国家863项目子课题。在《Biosystems Engineering》《Computers and Electronics in Agriculture》等期刊发表论文60余篇，《Optical engineering》《Precision agriculture》《Biosystems Engineering》《Computers and Electronics in Agriculture等20余家杂志审稿人

  • Received Date: March 30, 2024
  • Available Online: May 10, 2024
  • Published Date: June 05, 2024
    Graphical Abstract
    • Abstract

  • Cow intelligent monitoring is an important link in large-scale dairy farming. Video analysis has the advantages of contactless, low-cost, and intelligent analysis, and has become a hot spot in the research of intelligent identification technology of large-scale dairy cows. Dairy cow target detection, target tracking, and the technologies of individual and behavior recognition are of great significance for large-scale dairy cow supervision. Lighting, day and night alternations, fence occlusion and mutual occlusion caused by large number of cows in complex breeding environment are serious factors affecting the intelligent monitoring of large-scale dairy cows. This paper summarized the depth models and practical application commonly used in cow intelligent monitoring. The problems and challenges faced in the current research were put forward. The analysis result showed that the attention mechanism and hybrid convolution were effective methods to improve the recognition accuracy of the model, and the lightweight modules were conducive to reducing the complexity and computation of the model. The factors that affected the current research to be practical were computational complexity, universality and accuracy. It is necessary to conduct specific analyses based on the dairy farming environment and the condition of dairy cows to continuously meet the needs of large-scale farming while applying this technology.

    • FullText(HTML)
    • References (73)
  • [1]
    宋怀波, 李振宇, 吕帅朝, 等. 基于部分亲和场的行走奶牛骨架提取模型[J]. 农业机械学报, 2020, 51(8): 203-213. doi: 10.6041/j.issn.1000-1298.2020.08.023
    [2]
    杨蜀秦, 刘杨启航, 王振, 等. 基于融合坐标信息的改进YOLO-v4模型识别奶牛面部[J]. 农业工程学报, 2021, 37(15): 129-135. doi: 10.11975/j.issn.1002-6819.2021.15.016
    [3]
    LI Z, SONG L, DUAN Y, et al. Basic motion behaviour recognition of dairy cows based on skeleton and hybrid convolution algorithms[J]. Computers and Electronics in Agriculture, 2022, 196: 106889-106893. doi: 10.1016/j.compag.2022.106889
    [4]
    LI Z, ZHANG Q, LV S, et al. Fusion of RGB, optical flow and skeleton features for the detection of lameness in dairy cows[J]. Biosystems Engineering, 2022, 218: 62-77. doi: 10.1016/j.biosystemseng.2022.03.006
    [5]
    YANG G, XU X, SONG L, et al. Automated measurement of dairy cows body size via 3D point cloud data analysis[J]. Computers and Electronics in Agriculture, 2022, 200: 107218-107226. doi: 10.1016/j.compag.2022.107218
    [6]
    王政, 宋怀波, 王云飞, 等. 奶牛运动行为智能监测研究进展与技术趋势[J]. 智慧农业(中英文), 2022, 4(2): 36-52.
    [7]
    张姝瑾, 许兴时, 邓洪兴, 等. 基于YOLO_v8n-seg-FCA-BiFPN的奶牛身体分割方法研究[J]. 农业机械学报, 2024, 50(3): 282-289. doi: 10.6041/j.issn.1000-1298.2024.03.028
    [8]
    HUA Z, WANG Z, XU X, et al. An effective PoseC3D model for typical action recognition of dairy cows based on skeleton features[J]. Computers and Electronics in Agriculture, 2023, 212: 108152-108161. doi: 10.1016/j.compag.2023.108152
    [9]
    郝玉胜, 林强, 王维兰, 等. 基于Wi-Fi无线感知技术的奶牛爬跨行为识别[J]. 农业工程学报, 2020, 36(19): 168-176. doi: 10.11975/j.issn.1002-6819.2020.19.019
    [10]
    WANG Z, HUA Z, WEN Y, et al. E-YOLO: Recognition of estrus cow based on improved YOLOv8n model[J]. Expert Systems with Applications, 2024, 238: 1-13.
    [11]
    宋怀波, 华志新, 马宝玲, 等. 基于SimCC-ShuffNetv2的轻量化奶牛关键点检测方法[J]. 农业机械学报, 2023, 54(10): 275-281. doi: 10.6041/j.issn.1000-1298.2023.10.027
    [12]
    宋怀波, 姜波, 吴倩, 等. 基于头颈部轮廓拟合直线斜率特征的奶牛跛行检测方法[J]. 农业工程学报, 2018, 34(15): 190-199. doi: 10.11975/j.issn.1002-6819.2018.15.024
    [13]
    宋怀波, 牛满堂, 姬存慧, 等. 基于视频分析的多目标奶牛反刍行为监测[J]. 农业工程学报, 2018, 34(18): 211-218. doi: 10.11975/j.issn.1002-6819.2018.18.026
    [14]
    宋怀波, 李通, 姜波, 等. 基于 Horn-Schunck 光流法的多目标反刍奶牛嘴部自动监测[J]. 农业工程学报, 2018, 34(10): 163-171. doi: 10.11975/j.issn.1002-6819.2018.10.020
    [15]
    刘渊, 杨泽林, 赵永军. 基于RFID的物联网技术在畜牧业中的应用[J]. 黑龙江畜牧兽医, 2012(16): 15-17.
    [16]
    宋怀波, 吴頔华, 阴旭强, 等. 基于 Lucas-Kanade 稀疏光流算法的奶牛呼吸行为检测[J]. 农业工程学报, 2019, 35(17): 215-224. doi: 10.11975/j.issn.1002-6819.2019.17.026
    [17]
    刘杰鑫, 姜波, 何东健, 等. 基于高斯混合模型与CNN的奶牛个体识别方法研究[J]. 计算机应用与软件, 2018, 18(10): 159-164. doi: 10.3969/j.issn.1000-386x.2018.10.029
    [18]
    TASSINARI P, BOVO M, BENNI S, et al. A computer vision approach based on deep learning for the detection of dairy cows in free stall barn[J]. Computers and Electronics in Agriculture, 2021, 182: 106030-106042. doi: 10.1016/j.compag.2021.106030
    [19]
    BHOLE A, UDMALE S S, FALZON O, et al. CORF3D contour maps with application to Holstein cattle recognition from RGB and thermal images[J]. Expert Systems with Applications, 2022, 192: 116354-116365. doi: 10.1016/j.eswa.2021.116354
    [20]
    ZHENG Z, ZHANG X, QIN L, et al. Cows’ legs tracking and lameness detection in dairy cattle using video analysis and Siamese neural networks[J]. Computers and Electronics in Agriculture, 2023, 205: 107618. doi: 10.1016/j.compag.2023.107618
    [21]
    ZHENG Z, QIN L. Pruned YOLO-tracker: An efficient multi-cows basic behavior recognition and tracking technique[J]. Computers and Electronics in Agriculture, 2023, 213: 108172-108181. doi: 10.1016/j.compag.2023.108172
    [22]
    何东健, 刘建敏, 熊虹婷, 等. 基于改进YOLO+v3模型的挤奶奶牛个体识别方法[J]. 农业机械学报, 2020, 51(4): 250-260. doi: 10.6041/j.issn.1000-1298.2020.04.029
    [23]
    WANG Y, XU X, WANG Z, et al. ShuffleNet-Triplet: A lightweight RE-identification network for dairy cows in natural scenes[J]. Computers and Electronics in Agriculture, 2023, 205: 107632-107646. doi: 10.1016/j.compag.2023.107632
    [24]
    WU D, WANG Y, HAN M, et al. Using a CNN-LSTM for basic behaviors detection of a single dairy cow in a complex environment[J]. Computers and Electronics in Agriculture, 2021, 182: 106016-106028. doi: 10.1016/j.compag.2021.106016
    [25]
    YIN X, WU D, SHANG Y, et al. Using an efficient Net-LSTM for the recognition of single cow’s motion behaviours in a complicated environment[J]. Computers and Electronics in Agriculture, 2020, 10: 57-67.
    [26]
    WANG Y, LI R, WANG Z, et al. E3D: An efficient 3D CNN for the recognition of dairy cow's basic motion behavior[J]. Computers and Electronics in Agriculture, 2023, 205: 107607-107616. doi: 10.1016/j.compag.2022.107607
    [27]
    BEZEN R, EDAN Y, HALACHMI I. Computer vision system for measuring individual cow feed intake using RGB-D camera and deep learning algorithms[J]. Computers and Electronics in Agriculture, 2020, 172: 104345-104356.
    [28]
    BELLO R, ZAWAWI A, SUFRIL A, et al. Image-based individual cow recognition using body patterns[J]. International Journal of Advanced Computer Science and Applications, 2020, 11(3): 92-98.
    [29]
    WANG Y, CHEN T, LI B, et al. Automatic identification and analysis of multi-object cattle rumination based on computer vision[J]. Journal of Animal Science and Technology, 2023, 65(3): 519-534. doi: 10.5187/jast.2022.e87
    [30]
    WANG Y, KANG X, CHU M, et al. Deep learning-based automatic dairy cow ocular surface temperature detection from thermal images[J]. Computers and Electronics in Agriculture, 2022, 202: 107429-107442. doi: 10.1016/j.compag.2022.107429
    [31]
    QIAO Y, GUO Y, HE D. Cattle body detection based on YOLOv5-ASFF for precision livestock farming[J]. Computers and Electronics in Agriculture, 2023, 204: 107579-107589. doi: 10.1016/j.compag.2022.107579
    [32]
    李昊玥, 陈桂芬, 裴傲. 基于改进 Mask R-CNN的奶牛个体识别方法研究[J]. 华南农业大学学报, 2020, 41(6): 161-168. doi: 10.7671/j.issn.1001-411X.202003030
    [33]
    ZHENG Z, LI J, QIN L. YOLO-BYTE: An efficient multi-object tracking algorithm for automatic monitoring of dairy cows[J]. Computers and Electronics in Agriculture, 2023, 209: 107857-107867. doi: 10.1016/j.compag.2023.107857
    [34]
    MAKHURA O, WOODS J. Learn-select-track: An approach to multi-object tracking[J]. Signal Processing: Image Communication, 2019, 74: 153-161. doi: 10.1016/j.image.2019.02.009
    [35]
    LIANG Y, CHEN X. Tracking guided actions recognition for cows[J]. Quantitative Biology, 2022, 10: 351-365. doi: 10.15302/J-QB-022-0291
    [36]
    XU X, WANG Y, HUA Z, et al. Light-weight recognition network for dairy cows based on the fusion of YOLOv5s and channel pruning algorithm[J]. Transactions of the CSAE, 2023, 39(15): 153-163.
    [37]
    毛燕茹, 牛童, 王鹏, 等. 利用Kalman滤波和Hungarian算法的多目标奶牛嘴部跟踪及反刍监测[J]. 农业工程学报, 2021, 37(19): 192-201. doi: 10.11975/j.issn.1002-6819.2021.19.022
    [38]
    宋怀波, 阴旭强, 吴頔华, 等. 基于自适应无参核密度估计算法的运动奶牛目标检测[J]. 农业机械学报, 2019, 50(5): 45-58. doi: 10.6041/j.issn.1000-1298.2019.05.023
    [39]
    刘冬, 赵凯旋, 何东健. 基于混合高斯模型的移动奶牛目标实时检测[J]. 农业机械学报, 2016, 47(5): 288-294. doi: 10.6041/j.issn.1000-1298.2016.05.039
    [40]
    BELLO R, MOHAMED A, TALIB A, et al. Computer vision-based techniques for cow object recognition[C]// IOP Conference Series: Earth and Environmental Science. Bristol: IOP Publishing, 2021: 58-71.
    [41]
    LI W, JI Z, WANG L, et al. Automatic individual identification of Holstein dairy cows using tailhead images[J]. Computers and Electronics in Agriculture, 2017, 142: 622-631. doi: 10.1016/j.compag.2017.10.029
    [42]
    GUO Y, HE D, CHAI L. A machine vision-based method for monitoring scene-interactive behaviors of dairy calf[J]. Animals, 2020, 10(2): 190-204. doi: 10.3390/ani10020190
    [43]
    YU Z, LIU Y, YU S, et al. Teat detection of dairy cows based on deep learning neural network FS-YOLOv4 model[J]. Computers and Electronics in Agriculture, 2022, 200: 107224-107234. doi: 10.1016/j.compag.2022.107224
    [44]
    TIAN F, HU G, YU S, et al. An efficient multi-task convolutional neural network for dairy farm object detection and segmentation[J]. Computers and Electronics in Agriculture, 2023, 211: 108000-108017. doi: 10.1016/j.compag.2023.108000
    [45]
    MON S L, ZIN T T, TIN P, et al. Video-based automatic cattle identification system[C]// 2022 IEEE 11th Global Conference on Consumer Electronics, Osaka: IEEE, 2022: 490-491.
    [46]
    GUZHVA O, ARDÖ H, NILSSON M, et al. Now you see me: Convolutional neural network based tracker for dairy cows[J]. Frontiers in Robotics and AI, 2018, 5: 107-116. doi: 10.3389/frobt.2018.00107
    [47]
    张瑞红, 赵凯旋, 姬江涛, 等. 基于机器学习的奶牛颈环ID自动定位与识别方法[J]. 南京农业大学学报, 2021, 44(3): 586-595. doi: 10.7685/jnau.202010005
    [48]
    DU Y, KOU Y, LI B, et al. Individual identification of dairy cows based on deep learning and feature fusion[J]. Animal Science Journal, 2022, 93(1): 13789-13797. doi: 10.1111/asj.13789
    [49]
    XIAO J, LIU G, WANG K, et al. Cow identification in free-stall barns based on an improved Mask R-CNN and an SVM[J]. Computers and Electronics in Agriculture, 2022, 194: 107638-107649.
    [50]
    ACHOUR B, BELKADI M, FILALI I, et al. Image analysis for individual identification and feeding behaviour monitoring of dairy cows based on convolutional neural networks (CNN)[J]. Biosystems Engineering, 2020, 198: 31-49. doi: 10.1016/j.biosystemseng.2020.07.019
    [51]
    TONG L, BO J, DIHUA W, et al. Tracking multiple target cows’ ruminant mouth areas using optical flow and inter-frame difference methods[J]. IEEE Access, 2019, 7: 18520-185531.
    [52]
    刘月峰, 刘博, 暴祥, 等. 基于超轻量化孪生网络的自然场景奶牛单目标跟踪方法[J]. 农业机械学报, 2023, 54(10): 282-293. doi: 10.6041/j.issn.1000-1298.2023.10.028
    [53]
    HAO W, ZHANG K, HAN M, et al. A novel Jinnan individual cattle recognition approach based on mutual attention learning scheme[J]. Expert Systems with Applications, 2023, 230: 120551-120562. doi: 10.1016/j.eswa.2023.120551
    [54]
    ANDREW W, GREATWOOD C, BURGHARDT T. Visual localisation and individual identification of holstein friesian cattle via deep learning[C]//2017 IEEE International Conference on Computer Vision Workshops (ICCVW). IEEE, 2017: 2850-2859.
    [55]
    WENG Z, MENG F, LIU S, et al. Cattle face recognition based on a two-branch convolutional neural network[J]. Computers and Electronics in Agriculture, 2022, 196: 106871-106882. doi: 10.1016/j.compag.2022.106871
    [56]
    ZHAO K, HE D. Recognition of individual dairy cattle based on convolutional neural networks[J]. Transactions of the Chinese Society of Agricultural Engineering, 2015, 31: 181-187.
    [57]
    ZIN T, PHYO C, HAMA H. Image technology based cow identificatio[C]//Proceedings of the International Multi Conference of Engineers and Computer Scientists 2018, Hong Kong, 2018: 320-323.
    [58]
    WANG R, GAO R, LI Q, et al. An ultra-lightweight method for individual identification of cow-back pattern images in an open image set[J]. Expert Systems with Applications, 2024, 15: 54-68.
    [59]
    ANDREW W, GAO J, MULLAN S, et al. Visual identification of individual Holstein-Friesian cattle via deep metric learning[J]. Computers and Electronics in Agriculture, 2021, 185: 106133-106144. doi: 10.1016/j.compag.2021.106133
    [60]
    LU Y, WENG Z, ZHENG Z, et al. Algorithm for cattle identification based on locating key area[J]. Expert Systems with Applications, 2023, 228: 120365-120378. doi: 10.1016/j.eswa.2023.120365
    [61]
    HU H, DAI B, SHEN W, et al. Cow identification based on fusion of deep parts features[J]. Biosystems Engineering, 2020, 192: 245-256. doi: 10.1016/j.biosystemseng.2020.02.001
    [62]
    ZHANG K, HAN S, WU J, et al. Early lameness detection in dairy cattle based on wearable gait analysis using semi-supervised LSTM-Autoencoder[J]. Computers and Electronics in Agriculture, 2023, 213: 108252-108267. doi: 10.1016/j.compag.2023.108252
    [63]
    KUMAR S, SINGH S K, SINGH R S, et al. Real-time recognition of cattle using animal biometrics[J]. Journal of Real-Time Image Processing, 2016, 13(3): 505-526.
    [64]
    王克俭, 孙奕飞, 司永胜, 等. 基于时空特征的奶牛视频行为识别[J]. 农业机械学报, 2023, 54(5): 261-358. doi: 10.6041/j.issn.1000-1298.2023.05.027
    [65]
    MA S, ZHANG Q, LI T, et al. Basic motion behavior recognition of single dairy cow based on improved Rexnet 3D network[J]. Computers and Electronics in Agriculture, 2022, 194: 106772-106281. doi: 10.1016/j.compag.2022.106772
    [66]
    JIANG B, SONG H, WANG H, et al. Dairy cow lameness detection using a back curvature feature[J]. Computers and Electronics in Agriculture, 2022, 194: 106729-106738. doi: 10.1016/j.compag.2022.106729
    [67]
    杜粤猛, 史慧, 高峰, 等. 基于姿态估计和关键点特征向量的奶牛跛行识别方法[J]. 华中农业大学学报, 2023, 42(5): 251-261.
    [68]
    SHOGO H, YOSHITAKA M, MASAFUMI M, et al. Leveraging computer vision-based pose estimation technique in dairy cows for objective mobility analysis and scoring system[J]. Computers and Electronics in Agriculture, 2024, 217: 108573-108582. doi: 10.1016/j.compag.2023.108573
    [69]
    SHEN W, CHENG F, ZHANg Y, et al. Automatic recognition of ingestive-related behaviors of dairy cows based on triaxial acceleration[J]. Information Processing in Agriculture, 2020, 8: 427-443.
    [70]
    任晓惠, 刘刚, 张淼, 等. 基于支持向量机分类模型的奶牛行为识别方法[J]. 农业机械学报, 2019, 50(S1): 290-296. doi: 10.6041/j.issn.1000-1298.2019.S0.045
    [71]
    GUO Y, ZHANG Z, HE D, et al. Detection of cow mounting behavior using region geometry and optical flow characteristics[J]. Computers and Electronics in Agriculture, 2019, 10: 48-62.
    [72]
    JIANG B, YIN X, SONG H. Single-stream long-term optical flow convolution network for action recognition of lameness dairy cow[J]. Computers and Electronics in Agriculture, 2020, 175: 105536-105563. doi: 10.1016/j.compag.2020.105536
    [73]
    史学伟, 司永胜, 韩宪忠, 等. 基于SE-R(2+1)D网络的自然环境下的奶牛行为识别[J]. 河北农业大学学报, 2023, 46(1): 97-109.
    • Related Articles

Catalog

    Get Citation
    PDF
    XML
    Article views (656) PDF downloads (62) Cited by()
    Turn off MathJax
    Article Contents
    Abstract
    References

    Address： No. 483 Wushan Road, Tianhe District, Guangzhou Editorial Department of Journal of South China Agricultural University

    Postal Code：510642

    Tel：(020) 85280069/38746672

    Sponsor/Sponsored by：South China Agricultural University

    Editor-in-chief：Xue Hongwei

    ISSN: 1001-411X CN: 44-1110/S

    Email Alert
    RSS

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return