Research progress of intelligent monitoring technology for large-scale dairy cows based on video analysis
-
摘要:
奶牛智能监测是规模化奶牛养殖的重要环节,视频分析技术具备无接触、低成本及智能分析优势,已成为当前规模化奶牛智能监测技术研究的热点。奶牛目标检测、目标跟踪以及个体和行为识别技术对规模化奶牛监管具有重要意义,复杂养殖环境中的光照、昼夜交替变化、围栏遮挡以及牛群数量繁多导致的相互遮挡是影响规模化奶牛智能监测的重要因素。本文对基于视频分析的奶牛智能监测技术研究中常用的深度模型及应用情况进行综述,提出了当前研究中面临的问题与挑战。分析发现,注意力机制、混合卷积等技术是提高模型识别准确率的有效方法,轻量化模块有利于减少模型的复杂度与计算量;计算复杂度、普适性、准确性等是影响该技术推广应用的因素;具体应用时,需要针对奶牛养殖环境、奶牛状况等进行具体分析以不断满足规模化养殖的需求。
Abstract:Cow intelligent monitoring is an important link in large-scale dairy farming. Video analysis has the advantages of contactless, low-cost, and intelligent analysis, and has become a hot spot in the research of intelligent identification technology of large-scale dairy cows. Dairy cow target detection, target tracking, and the technologies of individual and behavior recognition are of great significance for large-scale dairy cow supervision. Lighting, day and night alternations, fence occlusion and mutual occlusion caused by large number of cows in complex breeding environment are serious factors affecting the intelligent monitoring of large-scale dairy cows. This paper summarized the depth models and practical application commonly used in cow intelligent monitoring. The problems and challenges faced in the current research were put forward. The analysis result showed that the attention mechanism and hybrid convolution were effective methods to improve the recognition accuracy of the model, and the lightweight modules were conducive to reducing the complexity and computation of the model. The factors that affected the current research to be practical were computational complexity, universality and accuracy. It is necessary to conduct specific analyses based on the dairy farming environment and the condition of dairy cows to continuously meet the needs of large-scale farming while applying this technology.
-
-
表 1 奶牛目标检测相关研究
Table 1 Research on target recognition of dairy cow
方法 Method 技术要点 Technical essential 结果 Result 文献 Literatrue 传统图像处理
Traditional image
processing基于递归背景建模思想 模型精度最高为95.00% [27] 基于核相关滤波算法 平均误检率为7.72% [13] 高斯混合模型及卷积神经网络相结合 准确率为99.81% [22] 基于Horn-Schunck光流法 检测充盈率为98.51% [14] 基于无参核密度估计背景建模方法 识别准确率为95.65% [28] 基于深度学习模型
Methods based on deep
learning model人工设计的四层卷积神经网络 准确率为89.95% [29] 利用RGB-D图像训练卷积神经网络 识别准确率为93.65% [30] 改进Mask R-CNN模型 模型精度最高为100% [31] 基于粒子滤波算法 准确率为89.00% [32] 基于YOLO算法 准确率为66.00% [23] 基于YOLOv5模型 准确率为90.00% [33] 融合YOLO与核相关滤波器 准确率为95.00% [34] 设计YOLOv5-ASFF模型 准确率为96.20% [35] 利用CBAM注意力机制对GhostNet进行改进 检测准确率为94.86% [36] 对YOLOv5s网络进行通道剪枝 模型精度为99.50% [37] 表 2 奶牛目标跟踪相关研究
Table 2 Research on cow target tracking
方法 Method 技术要点 Technical essential 结果 Result 文献 Literatrue 传统图像处理
Traditional image
processing联合稠密光流和帧间差分法 跟踪准确率为89.12% [46] 提出了一种SiamFC的跟踪器 最高跟踪准确率为100.00% [47] 基于深度学习模型
Methods based on deep
learning modelYOLOv4模型结合 Kalman滤波和Hungarian算法 准确率为93.92% [48] 提出了YOLO-BYTE跟踪模型 多目标跟踪准确率为83.00% [49] YOLOv5检测器剪枝融合Cascaded-Buffered IoU 多目标跟踪准确率为86.10% [26] Siamese注意力算法 跟踪准确率为93.80% [25] 超轻量化孪生网络模型Siamese-Remo 单目标跟踪平均重合度为0.47 [50] 表 3 奶牛个体识别相关研究
Table 3 Research on individual recognition of dairy cow
方法 Method 技术要点 Technical essential 结果 Result 文献 Literatrue 传统个体识别方法
Traditional individual
recognition method基于LeNet-5模型 识别准确率为90.55% [53] 自主设计深度卷积神经网络结合SVM 单幅图像识别时间为30.74 s [54] 自主设计卷积神经网络 准确率为96.55% [55] 基于机器学习的奶牛颈环 ID 自动定位与识别 识别准确率为96.98% [56] VGG-16+SVM 准确率为99.48% [57] 基于深度学习的方法
Method based on
deep learningR-CNN检测模型 识别准确率为86.10% [58] DCNN结合SVM 准确率最高为97.01% [59] YOLO和SVM 准确率为98.36% [60] Mask-R-CNN和SVM网络 准确率为98.67% [61] 基于度量学习的方法
Method based on metric learningShuffleNetv2模型融合交叉熵损失和三元组损失函数 精度为73.30% [39] ResNet-50结合A-softmax损失 准确率为94.26% [62] 表 4 奶牛行为识别相关研究
Table 4 Research on cow behavior recognition
方法 Method 技术要点 Technical essential 结果 Result 文献 Literatrue 传统方法
Traditional
methods基于正态分布背景统计模型 准确率为93.89% [10] 基于支持向量机模型的奶牛行为识别 准确率为98.02% [64] 背景减除法融合SVM分类器 准确率为90.92% [65] Lucas-Kanade稀疏光流算法 准确率为98.58% [21] 三轴加速度传感器 识别精度和召回率分别为
92.80%和95.60%[66] 基于深度学习的方法
Methods based on
deep learning model基于CNN-LSTM算法 准确率、召回率和特异性分别为
97.10%、96.50%和98.30%[40] 空间特征网络优化的Efficient-LSTM算法 识别精度为97.87% [41] 利用3D卷积对RexNet网络进行改进 准确率为95.00% [62] 半监督长短期记忆早期跛行自编码器算法 准确率为97.78% [67] 深度可分离卷积和3D卷积操作构建E3D的
奶牛基本运动行为识别模型准确率为98.17% [42] 基于姿态估计和膝关节角度特征向量的奶
牛跛行识别方法准确率为97.22% [68] E-YOLO的奶牛发情行为识别模型 准确率为93.90% [12] -
[1] 宋怀波, 李振宇, 吕帅朝, 等. 基于部分亲和场的行走奶牛骨架提取模型[J]. 农业机械学报, 2020, 51(8): 203-213. doi: 10.6041/j.issn.1000-1298.2020.08.023 [2] 杨蜀秦, 刘杨启航, 王振, 等. 基于融合坐标信息的改进YOLO-v4模型识别奶牛面部[J]. 农业工程学报, 2021, 37(15): 129-135. doi: 10.11975/j.issn.1002-6819.2021.15.016 [3] LI Z, SONG L, DUAN Y, et al. Basic motion behaviour recognition of dairy cows based on skeleton and hybrid convolution algorithms[J]. Computers and Electronics in Agriculture, 2022, 196: 106889-106893. doi: 10.1016/j.compag.2022.106889
[4] LI Z, ZHANG Q, LV S, et al. Fusion of RGB, optical flow and skeleton features for the detection of lameness in dairy cows[J]. Biosystems Engineering, 2022, 218: 62-77. doi: 10.1016/j.biosystemseng.2022.03.006
[5] YANG G, XU X, SONG L, et al. Automated measurement of dairy cows body size via 3D point cloud data analysis[J]. Computers and Electronics in Agriculture, 2022, 200: 107218-107226. doi: 10.1016/j.compag.2022.107218
[6] 王政, 宋怀波, 王云飞, 等. 奶牛运动行为智能监测研究进展与技术趋势[J]. 智慧农业(中英文), 2022, 4(2): 36-52. [7] 张姝瑾, 许兴时, 邓洪兴, 等. 基于YOLO_v8n-seg-FCA-BiFPN的奶牛身体分割方法研究[J]. 农业机械学报, 2024, 50(3): 282-289. doi: 10.6041/j.issn.1000-1298.2024.03.028 [8] HUA Z, WANG Z, XU X, et al. An effective PoseC3D model for typical action recognition of dairy cows based on skeleton features[J]. Computers and Electronics in Agriculture, 2023, 212: 108152-108161. doi: 10.1016/j.compag.2023.108152
[9] 郝玉胜, 林强, 王维兰, 等. 基于Wi-Fi无线感知技术的奶牛爬跨行为识别[J]. 农业工程学报, 2020, 36(19): 168-176. doi: 10.11975/j.issn.1002-6819.2020.19.019 [10] WANG Z, HUA Z, WEN Y, et al. E-YOLO: Recognition of estrus cow based on improved YOLOv8n model[J]. Expert Systems with Applications, 2024, 238: 1-13.
[11] 宋怀波, 华志新, 马宝玲, 等. 基于SimCC-ShuffNetv2的轻量化奶牛关键点检测方法[J]. 农业机械学报, 2023, 54(10): 275-281. doi: 10.6041/j.issn.1000-1298.2023.10.027 [12] 宋怀波, 姜波, 吴倩, 等. 基于头颈部轮廓拟合直线斜率特征的奶牛跛行检测方法[J]. 农业工程学报, 2018, 34(15): 190-199. doi: 10.11975/j.issn.1002-6819.2018.15.024 [13] 宋怀波, 牛满堂, 姬存慧, 等. 基于视频分析的多目标奶牛反刍行为监测[J]. 农业工程学报, 2018, 34(18): 211-218. doi: 10.11975/j.issn.1002-6819.2018.18.026 [14] 宋怀波, 李通, 姜波, 等. 基于 Horn-Schunck 光流法的多目标反刍奶牛嘴部自动监测[J]. 农业工程学报, 2018, 34(10): 163-171. doi: 10.11975/j.issn.1002-6819.2018.10.020 [15] 刘渊, 杨泽林, 赵永军. 基于RFID的物联网技术在畜牧业中的应用[J]. 黑龙江畜牧兽医, 2012(16): 15-17. [16] 宋怀波, 吴頔华, 阴旭强, 等. 基于 Lucas-Kanade 稀疏光流算法的奶牛呼吸行为检测[J]. 农业工程学报, 2019, 35(17): 215-224. doi: 10.11975/j.issn.1002-6819.2019.17.026 [17] 刘杰鑫, 姜波, 何东健, 等. 基于高斯混合模型与CNN的奶牛个体识别方法研究[J]. 计算机应用与软件, 2018, 18(10): 159-164. doi: 10.3969/j.issn.1000-386x.2018.10.029 [18] TASSINARI P, BOVO M, BENNI S, et al. A computer vision approach based on deep learning for the detection of dairy cows in free stall barn[J]. Computers and Electronics in Agriculture, 2021, 182: 106030-106042. doi: 10.1016/j.compag.2021.106030
[19] BHOLE A, UDMALE S S, FALZON O, et al. CORF3D contour maps with application to Holstein cattle recognition from RGB and thermal images[J]. Expert Systems with Applications, 2022, 192: 116354-116365. doi: 10.1016/j.eswa.2021.116354
[20] ZHENG Z, ZHANG X, QIN L, et al. Cows’ legs tracking and lameness detection in dairy cattle using video analysis and Siamese neural networks[J]. Computers and Electronics in Agriculture, 2023, 205: 107618. doi: 10.1016/j.compag.2023.107618
[21] ZHENG Z, QIN L. Pruned YOLO-tracker: An efficient multi-cows basic behavior recognition and tracking technique[J]. Computers and Electronics in Agriculture, 2023, 213: 108172-108181. doi: 10.1016/j.compag.2023.108172
[22] 何东健, 刘建敏, 熊虹婷, 等. 基于改进YOLO+v3模型的挤奶奶牛个体识别方法[J]. 农业机械学报, 2020, 51(4): 250-260. doi: 10.6041/j.issn.1000-1298.2020.04.029 [23] WANG Y, XU X, WANG Z, et al. ShuffleNet-Triplet: A lightweight RE-identification network for dairy cows in natural scenes[J]. Computers and Electronics in Agriculture, 2023, 205: 107632-107646. doi: 10.1016/j.compag.2023.107632
[24] WU D, WANG Y, HAN M, et al. Using a CNN-LSTM for basic behaviors detection of a single dairy cow in a complex environment[J]. Computers and Electronics in Agriculture, 2021, 182: 106016-106028. doi: 10.1016/j.compag.2021.106016
[25] YIN X, WU D, SHANG Y, et al. Using an efficient Net-LSTM for the recognition of single cow’s motion behaviours in a complicated environment[J]. Computers and Electronics in Agriculture, 2020, 10: 57-67.
[26] WANG Y, LI R, WANG Z, et al. E3D: An efficient 3D CNN for the recognition of dairy cow's basic motion behavior[J]. Computers and Electronics in Agriculture, 2023, 205: 107607-107616. doi: 10.1016/j.compag.2022.107607
[27] BEZEN R, EDAN Y, HALACHMI I. Computer vision system for measuring individual cow feed intake using RGB-D camera and deep learning algorithms[J]. Computers and Electronics in Agriculture, 2020, 172: 104345-104356.
[28] BELLO R, ZAWAWI A, SUFRIL A, et al. Image-based individual cow recognition using body patterns[J]. International Journal of Advanced Computer Science and Applications, 2020, 11(3): 92-98.
[29] WANG Y, CHEN T, LI B, et al. Automatic identification and analysis of multi-object cattle rumination based on computer vision[J]. Journal of Animal Science and Technology, 2023, 65(3): 519-534. doi: 10.5187/jast.2022.e87
[30] WANG Y, KANG X, CHU M, et al. Deep learning-based automatic dairy cow ocular surface temperature detection from thermal images[J]. Computers and Electronics in Agriculture, 2022, 202: 107429-107442. doi: 10.1016/j.compag.2022.107429
[31] QIAO Y, GUO Y, HE D. Cattle body detection based on YOLOv5-ASFF for precision livestock farming[J]. Computers and Electronics in Agriculture, 2023, 204: 107579-107589. doi: 10.1016/j.compag.2022.107579
[32] 李昊玥, 陈桂芬, 裴傲. 基于改进 Mask R-CNN的奶牛个体识别方法研究[J]. 华南农业大学学报, 2020, 41(6): 161-168. doi: 10.7671/j.issn.1001-411X.202003030 [33] ZHENG Z, LI J, QIN L. YOLO-BYTE: An efficient multi-object tracking algorithm for automatic monitoring of dairy cows[J]. Computers and Electronics in Agriculture, 2023, 209: 107857-107867. doi: 10.1016/j.compag.2023.107857
[34] MAKHURA O, WOODS J. Learn-select-track: An approach to multi-object tracking[J]. Signal Processing: Image Communication, 2019, 74: 153-161. doi: 10.1016/j.image.2019.02.009
[35] LIANG Y, CHEN X. Tracking guided actions recognition for cows[J]. Quantitative Biology, 2022, 10: 351-365. doi: 10.15302/J-QB-022-0291
[36] XU X, WANG Y, HUA Z, et al. Light-weight recognition network for dairy cows based on the fusion of YOLOv5s and channel pruning algorithm[J]. Transactions of the CSAE, 2023, 39(15): 153-163.
[37] 毛燕茹, 牛童, 王鹏, 等. 利用Kalman滤波和Hungarian算法的多目标奶牛嘴部跟踪及反刍监测[J]. 农业工程学报, 2021, 37(19): 192-201. doi: 10.11975/j.issn.1002-6819.2021.19.022 [38] 宋怀波, 阴旭强, 吴頔华, 等. 基于自适应无参核密度估计算法的运动奶牛目标检测[J]. 农业机械学报, 2019, 50(5): 45-58. doi: 10.6041/j.issn.1000-1298.2019.05.023 [39] 刘冬, 赵凯旋, 何东健. 基于混合高斯模型的移动奶牛目标实时检测[J]. 农业机械学报, 2016, 47(5): 288-294. doi: 10.6041/j.issn.1000-1298.2016.05.039 [40] BELLO R, MOHAMED A, TALIB A, et al. Computer vision-based techniques for cow object recognition[C]// IOP Conference Series: Earth and Environmental Science. Bristol: IOP Publishing, 2021: 58-71.
[41] LI W, JI Z, WANG L, et al. Automatic individual identification of Holstein dairy cows using tailhead images[J]. Computers and Electronics in Agriculture, 2017, 142: 622-631. doi: 10.1016/j.compag.2017.10.029
[42] GUO Y, HE D, CHAI L. A machine vision-based method for monitoring scene-interactive behaviors of dairy calf[J]. Animals, 2020, 10(2): 190-204. doi: 10.3390/ani10020190
[43] YU Z, LIU Y, YU S, et al. Teat detection of dairy cows based on deep learning neural network FS-YOLOv4 model[J]. Computers and Electronics in Agriculture, 2022, 200: 107224-107234. doi: 10.1016/j.compag.2022.107224
[44] TIAN F, HU G, YU S, et al. An efficient multi-task convolutional neural network for dairy farm object detection and segmentation[J]. Computers and Electronics in Agriculture, 2023, 211: 108000-108017. doi: 10.1016/j.compag.2023.108000
[45] MON S L, ZIN T T, TIN P, et al. Video-based automatic cattle identification system[C]// 2022 IEEE 11th Global Conference on Consumer Electronics, Osaka: IEEE, 2022: 490-491.
[46] GUZHVA O, ARDÖ H, NILSSON M, et al. Now you see me: Convolutional neural network based tracker for dairy cows[J]. Frontiers in Robotics and AI, 2018, 5: 107-116. doi: 10.3389/frobt.2018.00107
[47] 张瑞红, 赵凯旋, 姬江涛, 等. 基于机器学习的奶牛颈环ID自动定位与识别方法[J]. 南京农业大学学报, 2021, 44(3): 586-595. doi: 10.7685/jnau.202010005 [48] DU Y, KOU Y, LI B, et al. Individual identification of dairy cows based on deep learning and feature fusion[J]. Animal Science Journal, 2022, 93(1): 13789-13797. doi: 10.1111/asj.13789
[49] XIAO J, LIU G, WANG K, et al. Cow identification in free-stall barns based on an improved Mask R-CNN and an SVM[J]. Computers and Electronics in Agriculture, 2022, 194: 107638-107649.
[50] ACHOUR B, BELKADI M, FILALI I, et al. Image analysis for individual identification and feeding behaviour monitoring of dairy cows based on convolutional neural networks (CNN)[J]. Biosystems Engineering, 2020, 198: 31-49. doi: 10.1016/j.biosystemseng.2020.07.019
[51] TONG L, BO J, DIHUA W, et al. Tracking multiple target cows’ ruminant mouth areas using optical flow and inter-frame difference methods[J]. IEEE Access, 2019, 7: 18520-185531.
[52] 刘月峰, 刘博, 暴祥, 等. 基于超轻量化孪生网络的自然场景奶牛单目标跟踪方法[J]. 农业机械学报, 2023, 54(10): 282-293. doi: 10.6041/j.issn.1000-1298.2023.10.028 [53] HAO W, ZHANG K, HAN M, et al. A novel Jinnan individual cattle recognition approach based on mutual attention learning scheme[J]. Expert Systems with Applications, 2023, 230: 120551-120562. doi: 10.1016/j.eswa.2023.120551
[54] ANDREW W, GREATWOOD C, BURGHARDT T. Visual localisation and individual identification of holstein friesian cattle via deep learning[C]//2017 IEEE International Conference on Computer Vision Workshops (ICCVW). IEEE, 2017: 2850-2859.
[55] WENG Z, MENG F, LIU S, et al. Cattle face recognition based on a two-branch convolutional neural network[J]. Computers and Electronics in Agriculture, 2022, 196: 106871-106882. doi: 10.1016/j.compag.2022.106871
[56] ZHAO K, HE D. Recognition of individual dairy cattle based on convolutional neural networks[J]. Transactions of the Chinese Society of Agricultural Engineering, 2015, 31: 181-187.
[57] ZIN T, PHYO C, HAMA H. Image technology based cow identificatio[C]//Proceedings of the International Multi Conference of Engineers and Computer Scientists 2018, Hong Kong, 2018: 320-323.
[58] WANG R, GAO R, LI Q, et al. An ultra-lightweight method for individual identification of cow-back pattern images in an open image set[J]. Expert Systems with Applications, 2024, 15: 54-68.
[59] ANDREW W, GAO J, MULLAN S, et al. Visual identification of individual Holstein-Friesian cattle via deep metric learning[J]. Computers and Electronics in Agriculture, 2021, 185: 106133-106144. doi: 10.1016/j.compag.2021.106133
[60] LU Y, WENG Z, ZHENG Z, et al. Algorithm for cattle identification based on locating key area[J]. Expert Systems with Applications, 2023, 228: 120365-120378. doi: 10.1016/j.eswa.2023.120365
[61] HU H, DAI B, SHEN W, et al. Cow identification based on fusion of deep parts features[J]. Biosystems Engineering, 2020, 192: 245-256. doi: 10.1016/j.biosystemseng.2020.02.001
[62] ZHANG K, HAN S, WU J, et al. Early lameness detection in dairy cattle based on wearable gait analysis using semi-supervised LSTM-Autoencoder[J]. Computers and Electronics in Agriculture, 2023, 213: 108252-108267. doi: 10.1016/j.compag.2023.108252
[63] KUMAR S, SINGH S K, SINGH R S, et al. Real-time recognition of cattle using animal biometrics[J]. Journal of Real-Time Image Processing, 2016, 13(3): 505-526.
[64] 王克俭, 孙奕飞, 司永胜, 等. 基于时空特征的奶牛视频行为识别[J]. 农业机械学报, 2023, 54(5): 261-358. doi: 10.6041/j.issn.1000-1298.2023.05.027 [65] MA S, ZHANG Q, LI T, et al. Basic motion behavior recognition of single dairy cow based on improved Rexnet 3D network[J]. Computers and Electronics in Agriculture, 2022, 194: 106772-106281. doi: 10.1016/j.compag.2022.106772
[66] JIANG B, SONG H, WANG H, et al. Dairy cow lameness detection using a back curvature feature[J]. Computers and Electronics in Agriculture, 2022, 194: 106729-106738. doi: 10.1016/j.compag.2022.106729
[67] 杜粤猛, 史慧, 高峰, 等. 基于姿态估计和关键点特征向量的奶牛跛行识别方法[J]. 华中农业大学学报, 2023, 42(5): 251-261. [68] SHOGO H, YOSHITAKA M, MASAFUMI M, et al. Leveraging computer vision-based pose estimation technique in dairy cows for objective mobility analysis and scoring system[J]. Computers and Electronics in Agriculture, 2024, 217: 108573-108582. doi: 10.1016/j.compag.2023.108573
[69] SHEN W, CHENG F, ZHANg Y, et al. Automatic recognition of ingestive-related behaviors of dairy cows based on triaxial acceleration[J]. Information Processing in Agriculture, 2020, 8: 427-443.
[70] 任晓惠, 刘刚, 张淼, 等. 基于支持向量机分类模型的奶牛行为识别方法[J]. 农业机械学报, 2019, 50(S1): 290-296. doi: 10.6041/j.issn.1000-1298.2019.S0.045 [71] GUO Y, ZHANG Z, HE D, et al. Detection of cow mounting behavior using region geometry and optical flow characteristics[J]. Computers and Electronics in Agriculture, 2019, 10: 48-62.
[72] JIANG B, YIN X, SONG H. Single-stream long-term optical flow convolution network for action recognition of lameness dairy cow[J]. Computers and Electronics in Agriculture, 2020, 175: 105536-105563. doi: 10.1016/j.compag.2020.105536
[73] 史学伟, 司永胜, 韩宪忠, 等. 基于SE-R(2+1)D网络的自然环境下的奶牛行为识别[J]. 河北农业大学学报, 2023, 46(1): 97-109. -
期刊类型引用(4)
1. 邓文林. 乡土阔叶树种示范林建设——以韶关市国有仁化林场为例. 农村科学实验. 2024(18): 133-135 . 百度学术
2. 何潇,雷相东,段光爽,丰庆荣,张逸如,冯林艳. 气候变化对落叶松人工林生物量生长的影响模拟. 南京林业大学学报(自然科学版). 2023(03): 120-128 . 百度学术
3. 康梓杭,倪苗,邓晶,吴庆书. 海口市10种行道树最优生长模型研究. 热带生物学报. 2020(01): 79-83 . 百度学术
4. 曹磊,刘晓彤,李海奎,雷渊才. 广东省常绿阔叶林生物量生长模型. 林业科学研究. 2020(05): 61-67 . 百度学术
其他类型引用(9)