Instance segmentation of group-housed pigs based on recurrent residual attention

LIU Kun; YANG Huaiqing; YANG Hua; HU Zhiwei; MENG Ke

doi:10.7671/j.issn.1001-411X.202006013

Journal of South China Agricultural University > 2020 > 41(6): 169-178. > DOI: 10.7671/j.issn.1001-411X.202006013

LIU Kun, YANG Huaiqing, YANG Hua, et al. Instance segmentation of group-housed pigs based on recurrent residual attention[J]. Journal of South China Agricultural University, 2020, 41(6): 169-178. DOI: 10.7671/j.issn.1001-411X.202006013

Citation:

PDF (1391 KB)

Instance segmentation of group-housed pigs based on recurrent residual attention

1.
College of Information Science and Engineering, Shanxi Agricultural University, Taigu 030801, China
2.
Urban and Rural Planning of Architecture Department, Lüliang University, Lüliang 033001, China

More Information

Received Date: June 04, 2020
Available Online: May 17, 2023

Graphical Abstract

Abstract

Abstract

Objective
To realize high-precision segmentation of individual pigs under different conditions such as pig adhesion and debris shielding in a group breeding environment.

Method
A total of 45 group-housed pigs of 20 to 105 days from eight sheds in real farming scenes were studied. Mobile camera images were used as data sources, and data enhancement operations such as changing brightness and adding Gaussian noise were performed to obtain 3 834 annotated pictures. We explored multiple models with the cross-combinations of two backbone networks ResNet50, ResNet101 and two mission networks Mask R-CNN, Cascade mask R-CNN. We also introduced the idea of recurrent residual attention (RRA) into the two major task network models to improve the feature extraction ability and segmentation accuracy of the model without significantly increasing the amount of calculation.

Result
Compared with Cascade mask R-CNN-ResNet50, Mask R-CNN-ResNet50 improved AP_0.5, AP_0.75, AP_0.5-0.95 and AP_{0.5-0.95-large} by 4.3%, 3.5%, 2.2% and 2.2% respectively. Different numbers of RRA modules were added to explore the impact on the prediction performance of each task model. The experiment showed that adding two RRA modules had the most obvious improvement effect on each task model.

Conclusion
The Mask R-CNN-ResNet50 model with two RRA modules can more accurately and effectively segment group-housed pigs under different scenes. The results can provide a model support for subsequent identification and behavior analysis of live pigs.
- recurrent residual attention,
- instance segmentation,
- image processing,
- Mask R-CNN,
- Cascade mask R-CNN

FullText(HTML)

References (36)

References

[1]	孙龙清, 李玥, 邹远炳, 等. 基于改进 Graph Cut 算法的生猪图像分割方法[J]. 农业工程学报, 2017, 33(16): 196-202.
[2]	朱伟兴, 纪滨, 秦锋. 基于伪球算子边缘模型的猪前景帧检测[J]. 农业工程学报, 2012, 28(12): 189-194.
[3]	GUO Y, ZHU W, JIAO P, et al. Foreground detection of group-housed pigs based on the combination of mixture of Gaussians using prediction mechanism and threshold segmentation[J]. Biosyst Eng, 2014, 125(3): 98-104.
[4]	刘波, 朱伟兴, 霍冠英. 生猪轮廓红外与光学图像的融合算法[J]. 农业工程学报, 2013, 29(17): 113-120.
[5]	KRIZHEVSKY A, SUTSKEVER I, HINTONG E. Imagenet classification with deep convolutional neural networks[J]. Commun ACM, 2017, 60(6): 84-90.
[6]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//IEEE. Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas: IEEE, 2016: 770-778.
[7]	燕红文, 刘振宇, 崔清亮, 等. 基于改进Tiny-YOLO模型的群养生猪脸部姿态检测[J]. 农业工程学报, 2019, 35(18): 169-179.
[8]	薛月菊, 朱勋沐, 郑婵, 等. 基于改进Faster R-CNN识别深度视频图像哺乳母猪姿态[J]. 农业工程学报, 2018(9): 189-196.
[9]	NASIRAHMADI A, STURM B, EDWARDS S A, et al. Deep learning and machine vision approaches for posture detection of individual pigs.[J]. Sensors, 2019, 19(17): 3738. doi: 10.3390/s19173738.
[10]	刘岩, 孙龙清, 罗冰, 等. 基于改进CNN的多目标生猪检测算法[J]. 农业机械学报, 2019(S1): 283-289.
[11]	沈明霞, 太猛, CEDRIC Okinda, 等. 基于深层卷积神经网络的初生仔猪目标实时检测方法[J]. 农业机械学报, 2019, 50(8): 270-279.
[12]	HANSEN M, SMITH M L, SMITH L, et al. Towards on-farm pig face recognition using convolutional neural networks[J]. Comput Ind, 2018, 98: 145-152. doi: 10.1016/j.compind.2018.02.016
[13]	ZHANG L, GRAY H, YE X, et al. Automatic individual pig detection and tracking in pig farms[J]. Sensors, 2019, 19(5): 1188.
[14]	SUN L, ZOU Y, LI Y, et al. Multi target pigs tracking loss correction algorithm based on Faster R-CNN[J]. Int J Agr Biol Eng, 2018, 11(5): 192-197. doi: 10.25165/j.ijabe.20181105.4232
[15]	薛月菊, 杨晓帆, 郑婵, 等. 基于隐马尔科夫模型的深度视频哺乳母猪高危动作识别[J]. 农业工程学报, 2019, 35(13): 184-190.
[16]	高云, 陈斌, 廖慧敏, 等. 群养猪侵略性行为的深度学习识别方法[J]. 农业工程学报, 2019, 35(23): 192-200.
[17]	TIAN M, GUO H, CHEN H, et al. Automated pig counting using deep learning[J]. Electron. Agric. 2019,163, 104840. doi: 10.1016/j.compag.2019.05.049.
[18]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition. San Francisco: IEEE, 2015: 3431-3440.
[19]	杨阿庆, 薛月菊, 黄华盛, 等. 基于全卷积网络的哺乳母猪图像分割[J]. 农业工程学报, 2017, 33(23): 219-225.
[20]	RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge[J]. Int J Comput Vision, 2015, 115(3): 211-252. doi: 10.1007/s11263-015-0816-y
[21]	YANG A, HUANG H, ZHU X, et al. Automatic recognition of sow nursing behaviour using deep learning-based segmentation and spatial and temporal features[J]. Biosyst Eng, 2018, 175: 133-145. doi: 10.1016/j.biosystemseng.2018.09.011
[22]	YANG A, HUANG H, ZHENG C, et al. High-accuracy image segmentation for lactating sows using a fully convolutional network[J]. Biosyst Eng, 2018, 176: 36-47. doi: 10.1016/j.biosystemseng.2018.10.005
[23]	PSOTAE T, MITTEK M, PEREZ L C, et al. Multi-pig part detection and association with a fully-convolutional network[J]. Sensors, 2019, 19(4): 852. doi: 10.3390/s19040852.
[24]	胡志伟, 杨华, 娄甜田, 等. 基于全卷积网络的生猪轮廓提取[J]. 华南农业大学学报, 2018, 39(6): 111-119.
[25]	RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks forbiomedical image segmentation[C]//MICCAI. International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: MICCAI, 2015: 234-241.
[26]	HE K, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]//IEEE. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE 2017: 2980-2988.
[27]	林相泽, 朱赛华, 张俊媛, 等. 基于迁移学习和Mask R-CNN的稻飞虱图像分类方法[J]. 农业机械学报, 2019, 50(7): 201-207.
[28]	杨长辉, 王卓, 熊龙烨, 等. 基于 Mask R-CNN 的复杂背景下柑橘树枝干识别与重建[J]. 农业机械学报, 2019, 50(8): 22-30.
[29]	李丹, 张凯锋, 李行健, 等. 基于Mask R-CNN的猪只爬跨行为识别[J]. 农业机械学报, 2019, 50(增刊1): 261-266.
[30]	高云, 郭继亮, 黎煊, 等. 基于深度学习的群猪图像实例分割方法[J]. 农业机械学报, 2019, 50(4): 179-187.
[31]	WANG F, JIANG M, QIAN C, et al. Residual attention network for image classification[C]//IEEE. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2017: 3156-3164.
[32]	YU C, WANG J, PENG C, et al. BiSeNet: Bilateral segmentation network for real-time semantic segmentation[C]//ECCV. Proceedings of the European Conference on Computer Vision (ECCV). Cham: Springer International Publishing, 2018: 325-341.
[33]	徐诚极, 王晓峰, 杨亚东. Attention-YOLO: 引入注意力机制的YOLO检测算法[J]. 计算机工程与应用, 2019, 55(6): 13-23.
[34]	REN S Q, HE K M, GIRSHICK R. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Trans Pattern Anal Mach Intell, 2017, 39(6): 1137-1149.
[35]	CAI Z W, NUNO V. Cascade R-CNN: Delving into high quality object detection.[C]//IEEE. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6154-6162.
[36]	REZATOFIGHI H, TSOI N, GWAK J Y, et al. Generalized intersection over union: A metric and a loss for bounding box regression[C]//IEEE. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 658-666.

Cited By

Get Citation

PDF

XML

Article views (1073) PDF downloads (1289)

Turn off MathJax

Article Contents

Abstract

References

Instance segmentation of group-housed pigs based on recurrent residual attention

Abstract

References

Catalog

Export File

Citation

Format

Content