基于循环残差注意力的群养生猪实例分割

刘坤; 杨怀卿; 杨华; 胡志伟; 孟珂

doi:10.7671/j.issn.1001-411X.202006013

基于循环残差注意力的群养生猪实例分割

刘坤^1,,
杨怀卿^1, ,,
杨华¹,
胡志伟¹,
孟珂²

1.
山西农业大学信息科学与工程学院，山西太谷 030801
2.
吕梁学院城乡规划系，山西吕梁 033001

基金项目: 国家自然科学基金 (31671571)；山西农业大学青年科技创新基金(2019027)；山西农业大学科技创新基金(2020BQ14)

详细信息

作者简介:
刘坤(1996—)，男，硕士研究生，E-mail: liukunabcd@126.com

通讯作者:
杨怀卿(1964—)，男，副教授，E-mail: yanghq@sxau.edu.cn

中图分类号: S828
计量
- 文章访问数: 01025
- HTML全文浏览量: 0
- PDF下载量: 01282
出版历程
- 收稿日期: 2020-06-04
- 网络出版日期: 2023-05-17
- 刊出日期: 2020-11-09

Instance segmentation of group-housed pigs based on recurrent residual attention

1.
College of Information Science and Engineering, Shanxi Agricultural University, Taigu 030801, China
2.
Urban and Rural Planning of Architecture Department, Lüliang University, Lüliang 033001, China

摘要

摘要:
目的
在群养环境下，实现生猪粘连、杂物遮挡等不同条件下生猪个体的高精度分割。

方法
对真实养殖场景下的8栏日龄20~105 d共45头群养生猪进行研究，以移动相机拍摄图像为数据源，并执行改变亮度、加入高斯噪声等数据增强操作获取标注图片3 834张。探究基于2个骨干网络ResNet50、ResNet101与2个任务网络Mask R-CNN、Cascade mask R-CNN交叉结合的多种模型，并将循环残差注意力(RRA)思想引入2个任务网络模型中，在不显著增加计算量的前提下提升模型特征提取能力、提高分割精度。

结果
选用Mask R-CNN-ResNet50比Cascade mask R-CNN-ResNet50在AP_0.5、AP_0.75、AP_0.5-0.95和AP_{0.5-0.95-large}指标上分别提升4.3%、3.5%、2.2%和2.2%；加入不同数量的RRA模块以探究其对各个任务模型预测性能影响，试验表明加入2个RRA模块后对各个任务模型的提升效果最为明显。

结论
加入2个RRA模块的Mask R-CNN-ResNet50模型可以更精确、有效地对不同场景群养生猪进行分割，为后续生猪身份识别与行为分析提供模型支撑。
- 循环残差注意力 /
- 实例分割 /
- 图像处理 /
- Mask R-CNN /
- Cascade mask R-CNN
Abstract:
Objective
To realize high-precision segmentation of individual pigs under different conditions such as pig adhesion and debris shielding in a group breeding environment.

Method
A total of 45 group-housed pigs of 20 to 105 days from eight sheds in real farming scenes were studied. Mobile camera images were used as data sources, and data enhancement operations such as changing brightness and adding Gaussian noise were performed to obtain 3 834 annotated pictures. We explored multiple models with the cross-combinations of two backbone networks ResNet50, ResNet101 and two mission networks Mask R-CNN, Cascade mask R-CNN. We also introduced the idea of recurrent residual attention (RRA) into the two major task network models to improve the feature extraction ability and segmentation accuracy of the model without significantly increasing the amount of calculation.

Result
Compared with Cascade mask R-CNN-ResNet50, Mask R-CNN-ResNet50 improved AP_0.5, AP_0.75, AP_0.5-0.95 and AP_{0.5-0.95-large} by 4.3%, 3.5%, 2.2% and 2.2% respectively. Different numbers of RRA modules were added to explore the impact on the prediction performance of each task model. The experiment showed that adding two RRA modules had the most obvious improvement effect on each task model.

Conclusion
The Mask R-CNN-ResNet50 model with two RRA modules can more accurately and effectively segment group-housed pigs under different scenes. The results can provide a model support for subsequent identification and behavior analysis of live pigs.
- recurrent residual attention /
- instance segmentation /
- image processing /
- Mask R-CNN /
- Cascade mask R-CNN

HTML全文

图 1 数据集展示及预处理

①表示对2 048×1 024图片与标注结果进行整体放缩操作，②表示对图片进行数据增强操作

Figure 1. Dataset display and preprocessing

① represents the overall zoom operation on the 2 048×1 024 picture and the annotation result，② represents the data enhancement operation on the picture

下载: 全尺寸图片幻灯片

图 2 Mask R-CNN与Cascade mask R-CNN模型对比图

I表示输入图像，Backbone表示骨干网络，Align表示ROIAlign，H表示网络头部，C表示分类结果，M表示分割结果，①②③分别表示IOU阈值为0.5、0.6和0.7的头部网络

Figure 2. Mask R-CNN and Cascade mask R-CNN model comparison chart

I represents the input image, Backbone represents the backbone network, Align represents the ROIAlign, H represents the network header, C represents the classification result, M represents the segmentation result, ①②③ represent the head networks with IOU thresholds of 0.5, 0.6 and 0.7 respectively

下载: 全尺寸图片幻灯片

图 3 传统FPN与加入RRA模块的FPN对比结构图

M2~M5分别表示第2~5阶段自顶向下组件卷积操作后的输出，P2~P5分别表示第2~5阶段自底向上与自顶向下组件融合后的输出，N表示循环的次数，Higher与Lower表示高阶与低阶特征，ReLU与Sigmoid表示激活函数，①和②分别表示循环残差簇与注意力组件块

Figure 3. Comparative structure diagram of traditional FPN and FPN added with RRA module

M2−M5 respectively represent the output after the top-down component convolution operation in the second to fifth stages，P2−P5 respectively represent the output after the bottom-up and top-down component fusion of the second to fifth stages，N represents the number of cycles，Higher and Lower represent high-order and low-order features respectively，ReLU and Sigmoid respectively represent activation functions，① and ② represent recurrent residual cluster and attention component blocks, respectively

下载: 全尺寸图片幻灯片

图 4 加入不同循环残差注意力(RRA)模块的Cascade mask R-CNN与Mask R-CNN模型测试集预测结果可视化展示

①~⑥表示各种模型预测结果差异较大的部位对应的生猪编号

Figure 4. Visual display of prediction results of Cascade mask R-CNN and Mask R-CNN model test sets with different recurrent residual attention(RRA) modules

①−⑥ indicate the serial numbers of pigs having the parts with large differences in the prediction results of various models

下载: 全尺寸图片幻灯片

图 5 加入不同循环残差注意力(RRA)模块的Mask R-CNN-ResNet50模型对不同场景数据分割可视化展示

①~⑤表示各种模型预测结果差异较大的部位对应的生猪编号，GT表示通过LabelMe标注的真实结果

Figure 5. Visualization of the segmentation of different scene data using the Mask R-CNN-ResNet50 model added with different recurrent residual attention(RRA) modules

①−⑤ indicate the serial numbers of pigs having the parts with large differences in the prediction results of various models, GT represents the real results marked by LabelMe

下载: 全尺寸图片幻灯片

表 1 不同骨干网络下不同任务网络分割的平均精度(AP)¹⁾

Table 1 Average precision(AP) for different task networks under different backbone networks

模型 Model	骨干网络 Backbone network	AP_0.50	AP_0.75	AP_0.50-0.95	AP_{0.50-0.95-large}
Mask R-CNN	ResNet50	0.885	0.714	0.582	0.605
Mask R-CNN	ResNet101	0.660	0.348	0.368	0.384
Cascade mask R-CNN	ResNet50	0.842	0.679	0.560	0.583
Cascade mask R-CNN	ResNet101	0.834	0.642	0.541	0.564
1)AP_x表示以x作为IOU阈值所对应的AP指标值　1)AP_x represents the AP index value using x as the IOU threshold

下载: 导出CSV

表 2 引入不同循环残差注意力(RRA)模块任务网络分割的平均精度(AP)¹⁾

Table 2 Average precision (AP) of task network segmentation with different recurrent residual attention (RRA) modules

模型 Model	骨干网络 Backbone network	RRA模块数量 No. of RRA modules	AP_0.50	AP_0.75	AP_0.50-0.95	AP_{0.50-0.95-large}
Mask R-CNN	ResNet50	0	0.885	0.714	0.582	0.605
		1	0.860	0.706	0.585	0.611
		2	0.892	0.769	0.636	0.660
		3	0.890	0.764	0.628	0.651
		4	0.887	0.752	0.618	0.642
	ResNet101	0	0.660	0.348	0.368	0.384
		1	0.646	0.364	0.350	0.365
		2	0.686	0.474	0.400	0.416
		3	0.684	0.461	0.396	0.409
		4	0.684	0.469	0.396	0.410
Cascade mask R-CNN	ResNet50	0	0.842	0.679	0.560	0.583
		1	0.835	0.700	0.579	0.603
		2	0.874	0.767	0.631	0.654
		3	0.873	0.749	0.623	0.647
		4	0.864	0.745	0.614	0.639
	ResNet101	0	0.834	0.642	0.541	0.564
		1	0.836	0.687	0.568	0.591
		2	0.885	0.772	0.634	0.660
		3	0.883	0.770	0.630	0.654
		4	0.873	0.756	0.617	0.641
1) AP_x表示以x作为IOU阈值所对应的AP指标值　1) AP_x represents the AP index value using x as the IOU threshold

下载: 导出CSV

参考文献(36)

[1]	孙龙清, 李玥, 邹远炳, 等. 基于改进 Graph Cut 算法的生猪图像分割方法[J]. 农业工程学报, 2017, 33(16): 196-202.
[2]	朱伟兴, 纪滨, 秦锋. 基于伪球算子边缘模型的猪前景帧检测[J]. 农业工程学报, 2012, 28(12): 189-194.
[3]	GUO Y, ZHU W, JIAO P, et al. Foreground detection of group-housed pigs based on the combination of mixture of Gaussians using prediction mechanism and threshold segmentation[J]. Biosyst Eng, 2014, 125(3): 98-104.
[4]	刘波, 朱伟兴, 霍冠英. 生猪轮廓红外与光学图像的融合算法[J]. 农业工程学报, 2013, 29(17): 113-120.
[5]	KRIZHEVSKY A, SUTSKEVER I, HINTONG E. Imagenet classification with deep convolutional neural networks[J]. Commun ACM, 2017, 60(6): 84-90.
[6]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//IEEE. Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas: IEEE, 2016: 770-778.
[7]	燕红文, 刘振宇, 崔清亮, 等. 基于改进Tiny-YOLO模型的群养生猪脸部姿态检测[J]. 农业工程学报, 2019, 35(18): 169-179.
[8]	薛月菊, 朱勋沐, 郑婵, 等. 基于改进Faster R-CNN识别深度视频图像哺乳母猪姿态[J]. 农业工程学报, 2018(9): 189-196.
[9]	NASIRAHMADI A, STURM B, EDWARDS S A, et al. Deep learning and machine vision approaches for posture detection of individual pigs.[J]. Sensors, 2019, 19(17): 3738. doi: 10.3390/s19173738.
[10]	刘岩, 孙龙清, 罗冰, 等. 基于改进CNN的多目标生猪检测算法[J]. 农业机械学报, 2019(S1): 283-289.
[11]	沈明霞, 太猛, CEDRIC Okinda, 等. 基于深层卷积神经网络的初生仔猪目标实时检测方法[J]. 农业机械学报, 2019, 50(8): 270-279.
[12]	HANSEN M, SMITH M L, SMITH L, et al. Towards on-farm pig face recognition using convolutional neural networks[J]. Comput Ind, 2018, 98: 145-152. doi: 10.1016/j.compind.2018.02.016
[13]	ZHANG L, GRAY H, YE X, et al. Automatic individual pig detection and tracking in pig farms[J]. Sensors, 2019, 19(5): 1188.
[14]	SUN L, ZOU Y, LI Y, et al. Multi target pigs tracking loss correction algorithm based on Faster R-CNN[J]. Int J Agr Biol Eng, 2018, 11(5): 192-197. doi: 10.25165/j.ijabe.20181105.4232
[15]	薛月菊, 杨晓帆, 郑婵, 等. 基于隐马尔科夫模型的深度视频哺乳母猪高危动作识别[J]. 农业工程学报, 2019, 35(13): 184-190.
[16]	高云, 陈斌, 廖慧敏, 等. 群养猪侵略性行为的深度学习识别方法[J]. 农业工程学报, 2019, 35(23): 192-200.
[17]	TIAN M, GUO H, CHEN H, et al. Automated pig counting using deep learning[J]. Electron. Agric. 2019,163, 104840. doi: 10.1016/j.compag.2019.05.049.
[18]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition. San Francisco: IEEE, 2015: 3431-3440.
[19]	杨阿庆, 薛月菊, 黄华盛, 等. 基于全卷积网络的哺乳母猪图像分割[J]. 农业工程学报, 2017, 33(23): 219-225.
[20]	RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge[J]. Int J Comput Vision, 2015, 115(3): 211-252. doi: 10.1007/s11263-015-0816-y
[21]	YANG A, HUANG H, ZHU X, et al. Automatic recognition of sow nursing behaviour using deep learning-based segmentation and spatial and temporal features[J]. Biosyst Eng, 2018, 175: 133-145. doi: 10.1016/j.biosystemseng.2018.09.011
[22]	YANG A, HUANG H, ZHENG C, et al. High-accuracy image segmentation for lactating sows using a fully convolutional network[J]. Biosyst Eng, 2018, 176: 36-47. doi: 10.1016/j.biosystemseng.2018.10.005
[23]	PSOTAE T, MITTEK M, PEREZ L C, et al. Multi-pig part detection and association with a fully-convolutional network[J]. Sensors, 2019, 19(4): 852. doi: 10.3390/s19040852.
[24]	胡志伟, 杨华, 娄甜田, 等. 基于全卷积网络的生猪轮廓提取[J]. 华南农业大学学报, 2018, 39(6): 111-119.
[25]	RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks forbiomedical image segmentation[C]//MICCAI. International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: MICCAI, 2015: 234-241.
[26]	HE K, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]//IEEE. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE 2017: 2980-2988.
[27]	林相泽, 朱赛华, 张俊媛, 等. 基于迁移学习和Mask R-CNN的稻飞虱图像分类方法[J]. 农业机械学报, 2019, 50(7): 201-207.
[28]	杨长辉, 王卓, 熊龙烨, 等. 基于 Mask R-CNN 的复杂背景下柑橘树枝干识别与重建[J]. 农业机械学报, 2019, 50(8): 22-30.
[29]	李丹, 张凯锋, 李行健, 等. 基于Mask R-CNN的猪只爬跨行为识别[J]. 农业机械学报, 2019, 50(增刊1): 261-266.
[30]	高云, 郭继亮, 黎煊, 等. 基于深度学习的群猪图像实例分割方法[J]. 农业机械学报, 2019, 50(4): 179-187.
[31]	WANG F, JIANG M, QIAN C, et al. Residual attention network for image classification[C]//IEEE. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2017: 3156-3164.
[32]	YU C, WANG J, PENG C, et al. BiSeNet: Bilateral segmentation network for real-time semantic segmentation[C]//ECCV. Proceedings of the European Conference on Computer Vision (ECCV). Cham: Springer International Publishing, 2018: 325-341.
[33]	徐诚极, 王晓峰, 杨亚东. Attention-YOLO: 引入注意力机制的YOLO检测算法[J]. 计算机工程与应用, 2019, 55(6): 13-23.
[34]	REN S Q, HE K M, GIRSHICK R. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Trans Pattern Anal Mach Intell, 2017, 39(6): 1137-1149.
[35]	CAI Z W, NUNO V. Cascade R-CNN: Delving into high quality object detection.[C]//IEEE. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6154-6162.
[36]	REZATOFIGHI H, TSOI N, GWAK J Y, et al. Generalized intersection over union: A metric and a loss for bounding box regression[C]//IEEE. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 658-666.

施引文献(17)

期刊类型引用(10)

1.	邓永涛，尹苗，王聪，陈希文. 猪舍环境影响因子预测模型的研究进展. 家畜生态学报. 2025(02): 122-128 . 百度学术
2.	刘艳昌，吴延昌，郭颖轩，李冬阳，张志霞，左现刚，李国厚. 生猪生长环境舒适度实时监测系统设计与实现. 家畜生态学报. 2024(06): 76-81+96 . 百度学术
3.	刘丙杰，侯慕馨，冀海燕. 基于支持向量机集成的船舶舱室温湿度预测. 海军工程大学学报. 2024(03): 21-25+32 . 百度学术
4.	朱佳明，孙彬，蒲施桦，潘学民，徐顺来，胡彬，齐仁立. 基于多环境因素分析的猪舍温湿度预测模型. 华南农业大学学报. 2024(05): 709-721 . 本站查看
5.	杨晓平，李雪寒，聂鹏程. 设施生猪环境调控现状研究和未来展望. 中国农业信息. 2024(02): 83-96 . 百度学术
6.	尤德安，邬锡权，陆畅，董钊杰，曾志雄，吕恩利. 数字孪生技术在现代化猪场生产应用的研究进展. 现代农业装备. 2023(04): 2-9+70 . 百度学术
7.	翟子鹤，陈小文，高莉平，张天柱. 基于梯度提升算法的温室黄瓜株高生长模拟. 中国农业大学学报. 2022(05): 134-145 . 百度学术
8.	杨东轩，吴叶兰，张刚刚，刘硕. 基于边缘计算与深度学习的禽舍监测系统设计. 江苏农业科学. 2022(09): 219-225 . 百度学术
9.	陈谦，杨涵，王宝刚，李文生，钱建平. 基于GRU神经网络模型的冷链运输温度时序预测. 农业大数据学报. 2022(01): 82-88 . 百度学术
10.	刘炫文，刘仁鑫. 猪舍小气候环境模拟预测技术研究综述. 江西农业学报. 2022(08): 122-126 . 百度学术