Research on fast recognition of banana multi-target features by visual robot in complex environment
-
摘要:目的
针对野外环境下断蕾机器人对多特征的变量目标快速识别难题，以及目标受到树叶、遮挡及光照影响精度的问题，提出多特征目标的快速识别方法。
方法提出对香蕉果实、果轴和花蕾这3个目标进行多尺度特征提取及模型分类，融合聚类算法设计新的目标候选框参数，提出改进YOLOv3模型及网络结构参数的YOLO-Banana模型；为了平衡速度和准确度，用YOLO-Banana和Faster R-CNN分别对变化尺寸的香蕉多目标进行试验，研究算法对识别精度与速度的影响。
结果YOLO-Banana和Faster R-CNN这2种算法识别香蕉、花蕾和果轴的总平均精度分别为91.03%和95.16%，平均每张图像识别所需时间分别为0.237和0.434 s。2种算法精度均高于90%，YOLO-Banana的速度相对快1.83倍，更符合实时作业的需求。
结论野外蕉园环境下，采用YOLO-Banana模型进行香蕉多目标识别，可满足断蕾机器人视觉识别的速度及精度要求。
Abstract:ObjectiveAiming at fast recognition of multi-feature variable target by robot vision in field environment, and considering the problem that the accuracy of target is affected by leaves, shade and light, a fast recognition method of multi-feature target was proposed.
MethodA multi-scale feature extraction and classification model was proposed for three targets including banana fruit, fruit axis and flower bud. New target candidate box parameters were designed using clustering algorithm. The network structure parameters of YOLOv3 model were optimized and the YOLO-Banana model was proposed. In order to balance the speed and accuracy, YOLO-Banana and Faster R-CNN were used to conduct experiments on banana multi-targets with varying sizes. The effects of algorithms on recognition accuracy and speed were analyzed.
ResultThe average accuracies of YOLO-Banana and Faster R-CNN algorithms on banana fruit, fruit axis and flower bud were 91.03% and 95.16% respectively, average recognition time per photo was 0.237 and 0.434 s respectively. Therefore the accuracies of two algorithms were both above 90%. YOLO-Banana had relatively 1.83 times faster speed than Faster R-CNN, better satisfying the requirement of real-time operation.
ConclusionIn the wild environment, utilization of YOLO-Banana model for banana multi-target recognition can meet the speed and accuracy requirements for visual recognition by the bud-breaking robots.
-
Keywords:
- Deep learning /
- Flower bud /
- Fruit axis /
- Multi-feature /
- Multi-target recognition /
- Fast classification
-
根据市场的发展方向，二区原种场不断进行猪优良品种选育改良，最终选育出繁殖特性突出的种猪，为温氏WS501配套系提供了优质的第一母本。本研究对该猪场2014—2017年繁殖相关的数据进行统计及分析，以期为优良品种选育提供科学依据。
1. 材料与方法
1.1 材料
WS501配套系W62种母猪和WS501配套系W62种公猪。
1.2 方法
调查并记录2014到2017年WS501配套系W62种母猪的繁殖特性，包括初配日龄、初配体重、日增重、开配情期、初胎产仔数、产仔数、产仔均重和年产胎次。
调查并记录2014到2017年WS501配套系W62种公猪的繁殖特性，包括精液量、精液密度及合格率。
1.3 数据处理
样本数据用EXCEL、Foxpro6.0、SPSS10.0等软件进行整理和统计分析。
2. 结果与分析
2.1 种母猪的繁殖特性
WS501配套系W62种母猪的初配日龄为225~260 d，平均初配日龄接近240 d，具体结果见表 1。初配体重大于125 kg，平均初配体重160 kg，从2014到2017年，增加了4.16 kg，详情见表 2。初配日增重是后备母猪初配体重除以初配日龄，日增重为480.0~1 076.2 g，平均日增重为666.67 g，详情见表 3。WS501配套系W62种母猪的初情期出现较早，一般在150~180日龄时出现第1个发情期，平均3.50个情期，详情见表 4。综上所述，WS501配套系W62种母猪长速快、发情早、情期多。
表 1 WS501配套系母系种猪的初配日龄d 年份 个体数 平均值 标准差 最大值 最小值 2014 778 257.91 20.19 350 219 2015 1 040 246.37 13.48 365 222 2016 1 132 244.33 13.39 362 214 2017 1 327 243.33 12.76 321 219 总计 4 277 246.99 15.62 365 214 表 2 WS501配套系母系种猪的初配体重kg 年份 个体数 平均值 标准差 最大值 最小值 2014 778 162.63 5.04 230 150 2015 1 040 160.02 5.55 260 125 2016 1 132 166.05 11.53 222 140 2017 1 327 166.79 11.64 212 140 总计 4 277 164.19 9.85 260 125 表 3 WS501配套系母系种猪的日增重g 年份 个体数 平均值 标准差 最大值 最小值 2014 778 633.72 46.72 946.50 480.00 2015 1 040 651.01 37.13 1 076.20 482.36 2016 1 132 680.57 47.69 831.99 548.79 2017 1 327 686.32 47.13 832.81 496.63 总计 4 277 666.67 49.51 1 076.20 480.00 表 4 WS501配套系母系种猪的开配情期d 年份 个体数 平均情期 标准差 最大值 最小值 2014 778 3.83 1.23 8 2 2015 1 040 3.34 0.84 9 2 2016 1 132 3.43 0.97 8 1 2017 1 327 3.55 0.97 7 1 总计 4 277 3.52 1.01 9 1 WS501配套系W62种母猪的平均总仔数13.92头(初胎13.81头)，平均活仔数12.72头(初胎12.65头)，平均健仔数10.65头(初胎10.51头)，总分娩率89.28%(初胎90.21%)，详情见表 5、表 6。仔猪平均出生重为1.28 kg，详情见表 7。平均无效生产日为41.87 d，年产胎次2.43胎，详情见表 8。综上所述，WS501配套系W62种母猪高产、年产胎次多。
表 5 WS501配套系母系种猪的产仔数年份 分娩窝数 分娩率/% 总仔1)/头 活仔1)/头 健仔1)/头 2014 3 445 88.49 13.85±3.69 12.76±3.64 10.43±2.95 2015 3 623 89.14 13.34±3.44 12.38±3.49 10.34±2.98 2016 3 485 88.89 14.11±3.31 12.64±3.41 10.38±2.98 2017 3 532 90.60 14.40±3.27 13.09±3.26 11.46±2.94 总计 14 085 89.28 13.92±3.45 12.72±3.46 10.65±3.00 1)数据为平均值±标准差 表 6 WS501配套系母系种猪的初胎产仔数年份 分娩窝数 分娩率/% 总仔1)/头 活仔1)/头 健仔1)/头 2014 849 89.65 13.92±3.42 12.89±3.49 10.36±2.94 2015 1 019 90.34 13.31±3.14 12.49±3.30 10.30±2.89 2016 1 030 89.19 13.85±3.20 12.29±3.44 9.97±3.06 2017 1 243 91.38 14.13±3.08 12.91±3.22 11.22±2.93 总计 4 141 90.21 13.81±3.21 12.65±3.36 10.51±3.00 1)数据为平均值±标准差 表 7 WS501配套系母系种猪的产仔均重年份 分娩
窝数窝产总仔
/头仔猪均重
/kg标准差
/kg2014 3 445 13.85 1.29 0.24 2015 3 623 13.34 1.23 0.22 2016 3 485 14.11 1.24 0.25 2017 3 532 14.40 1.35 0.25 总计 14 085 13.92 1.28 0.25 表 8 WS501配套系母系种猪的年产胎次年份 个体数 无效生
产日/d年产
胎次/窝2014 1 366 46.65 2.39 2015 1 393 40.66 2.47 2016 1 495 40.25 2.44 2017 1 442 42.29 2.39 平均值 1 424 41.87 2.43 2.2 种公猪的繁殖特性
WS501配套系W62种公猪，具有体型壮硕(高、长)、长速快、115 kg体重校正日龄小的特征，精液量适中、精液密度较优，平均精液量为每次298 mL，精液平均密度为2.87×108 mL-1，精液平均合格率为96.06%，具体数据见表 9。
表 9 WS501配套系母系种公猪精液质量年份 采精次数 精液量/mL 密度/(×108 mL-1) 合格率/% 2014 1 221 277.12±101.13 2.94±1.04 97.46 2015 1 441 276.15±87.32 2.64±0.67 94.38 2016 1 173 335.58±90.77 2.75±0.52 93.35 2017 1 634 307.49±98.01 3.12±0.90 98.53 总计 5 469 298.48±97.35 2.87±0.83 96.09 3. 结论与讨论
有研究表明丹系大白母猪的平均胎产活仔数为15.37头，美丹大白母猪的平均胎产活仔数为12.35头[1]，WS501配套系W62种母猪生产的平均活仔数为12.72头，比丹系大白少2.63头，比美系大白多0.37头，说明WS501配套系种母猪繁殖性能与丹系母猪还有一定的差距。潘英等[2]研究表明加系大白母猪在231~260日龄初配时繁殖性能最佳，WS501配套系种母猪也符合这一规律，平均初配日龄为247 d，同时通过长期选育可将最适初配日龄不断降低，以保证配套系的长速优势。根据丁月云等[3]的研究，丹系大白母猪初配平均日增重为602.4 g，李庆岗等[4]研究表明，美系大白母猪初配平均日增重为609.9 g，WS501配套系种母猪日增重为480.0~1 076.2 g，平均日增重为666.67 g，与丹系和美系大白母猪对比具有长速优势。
WS501配套系W62母系种猪的繁殖性能较好、长速快、产仔多，充分发挥了新法系大白母猪的繁殖特性，但是距离丹系大白种猪高产的繁殖特性，还有较长时间的育种过程。
-
![]()
图 1 顺、逆光下的香蕉果实、花蕾和果轴
Figure 1. Banana fruit，flower bud and fruit axis target detection under frontlight and backlight
![]()
图 2 基于香蕉多特征目标的YOLOv3网络结构
Figure 2. YOLOv3 network structure based on banana multi-feature target
![]()
图 4 顺光下香蕉测试结果对比图
a1, b1, c1表示香蕉原图；a2(1.00, 0.93, 0.99), b2(0.99, 0.92, 0.98), c2(1.00, 0.98, 1.00)表示YOLO-Banana模型处理后的香蕉图像；a3(1.00, 1.00, 1.00), b3(1.00, 1.00, 0.999), c3(1.00, 1.00, 0.999)表示 Faster R-CNN模型处理后的香蕉图像；括号内数字分别代表该算法对香蕉果实, 果轴和花蕾的置信度
Figure 4. Comparison of banana test results under frontlight
a1, b1 and c1 are original images of banana; a2(1.00, 0.93, 0.99), b2(0.99, 0.92, 0.98), c2(1.00, 0.98, 1.00) are banana images treated by YOLO-Banana model; a3(1.00, 1.00, 1.00), b3(1.00, 1.00, 0.999), c3(1.00, 1.00, 0.999) are banana images treated by Faster R-CNN model; The numbers in parentheses represent the confidence of the algorithm for banana fruit, fruit axis and flower bud, respectively
![]()
图 5 逆光下香蕉测试结果对比
a1表示香蕉原图；a2(0.97, 0.35, 0.99)表示YOLO-Banana模型处理后的香蕉图像；a3(1.00, 0.999, 0.999)表示 Faster R-CNN模型处理后的香蕉图像；括号内数字分别代表该算法对香蕉果实, 果轴和花蕾的置信度
Figure 5. Comparison of banana test results under backlight
a1 is original image of banana, a2 (0.97, 0.35, 0.99)is banana image treated by YOLO-Banana model; a3 (1.00, 0.999, 0.999) is banana image treated by Faster R-CNN model; The numbers in parentheses represent the confidence of the algorithm for banana fruit, fruit axis and flower bud, respectively
表 1 香蕉多目标候选框参数
Table 1 Parameters of banana multi target candidate box for banana
候选框参数 Parameters of candidate box 1 2 3 4 5 6 7 8 9 新候选框高 Height of new candidate box 23 25 28 31 43 51 66 169 172 新候选框宽 Width of new candidate box 31 53 81 148 67 87 111 233 158 原候选框高 Height of original candidate box 10 16 33 30 62 59 116 156 373 原候选框宽 Width of original candidate box 13 30 23 61 45 119 90 198 326 
下载: 导出CSV
表 2 YOLO-Banana与Faster R-CNN的香蕉多目标特征检测结果的对比
Table 2 Comparison of multi-target feature detection results for banana between YOLO-Banana and Faster R-CNN
模型
Model花蕾 Flower bud 果实 Fruit 果轴 Fruit axis 总平均精度/%
Mean of average
precisiont/s 平均精度/%
Average precision召回率/%
Recall平均精度/%
Average precision召回率/%
Recall平均精度/%
Average precision召回率/%
RecallFaster R-CNN 99 99.48 98 98.05 89 89.51 95.16 0.434 YOLO-Banana 97 97.38 95 95.12 81 85.19 91.03 0.237
下载: 导出CSV
表 3 YOLO-Banana与Faster R-CNN的顺光逆光下的香蕉多目标检测结果的对比
Table 3 Comparison of multi-target detection results for banana between YOLO-Banana and Faster R-CNN under frontlight and backlight conditions
% 模型
Model花蕾精度
Flower bud average precision果实精度
Fruit average precision果轴精度
Fruit axis average precision总平均精度
Mean of average precision顺光
Frontlight逆光
Backlight顺光
Frontlight逆光
Backlight顺光
Frontlight逆光
Backlight顺光
Frontlight逆光
BacklightFaster R-CNN 100 100 100 100 93 92 97.64 97.47 YOLO-Banana 100 98 100 93 93 83 97.58 91.24
下载: 导出CSV
-
[1] YIN Y H, LI H F, FU W. Faster-YOLO: An accurate and faster object detection method[J]. Digital Signal Processing, 2020, 102: 102756. doi: 10.1016/j.dsp.2020.102756.
[2] 何银水, 余卓骅, 李健, 等. 基于视觉特征的厚板机器人焊接焊缝轮廓的有效提取[J]. 机械工程学报, 2019, 55(17): 56-60. [3] 訾斌, 尹泽强, 李永昌, 等. 基于YOLO模型的柔索并联机器人移动构件快速定位方法[J]. 机械工程学报, 2019, 55(3): 64-72. [4] 孙国鹏, 郝向阳, 张振杰, 等. 多特征判断的合作目标识别方法[J]. 系统仿真学报, 2018, 30(6): 2377-2383. [5] 薛培林, 吴愿, 殷国栋, 等. 基于信息融合的城市自主车辆实时目标识别[J]. 机械工程学报, 2020, 56(12): 165-173. [6] 张青, 邹湘军, 林桂潮, 等. 草莓重量和形状图像特征提取与在线分级方法[J]. 系统仿真学报, 2019, 31(1): 7-15. [7] TANG Y C, LI L J, WANG C L, et al. Real-time detection of surface deformation and strain in recycled aggregate concrete-filled steel tubular columns via four-ocular vision[J]. Robotics and Computer-Integrated Manufacturing, 2019, 59: 36-46. doi: 10.1016/j.rcim.2019.03.001
[8] DAI Z, SONG H S, LIANG H X, et al. Traffic parameter estimation and control system based on machine vision[J]. Journal of Ambient Intelligence and Humanized Computing, 2020: 1-13. doi: 10.1007/s12652-020-02052-5.
[9] MIAN A S, BENNAMOUN M, OWENS R. Three-dimensional model-based object recognition and segmentation in cluttered scenes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(10): 1584-1601. doi: 10.1109/TPAMI.2006.213
[10] WEI X Q, JIA K, LAN J H, et al. Automatic method of fruit object extraction under complex agricultural background for vision system of fruit picking robot[J]. Optik, 2014, 125(19): 5684-5689. doi: 10.1016/j.ijleo.2014.07.001.
[11] GONGAL A, AMATYA S, KARKEE M, et al. Sensors and systems for fruit detection and localization: A review[J]. Computers and Electronics in Agriculture, 2015, 116: 8-19. doi: 10.1016/j.compag.2015.05.021.
[12] TANG Y C, CHEN M Y, WANG C L, et al. Recognition and localization methods for vision-based fruit picking robots: A review[J]. Frontiers in Plant Science, 2020, 11: 510. doi: 10.3389/fpls.2020.00510.
[13] FU L H, DUAN J L, ZOU X J, et al. Banana detection based on color and texture features in the natural environment[J]. Computers and Electronics in Agriculture, 2019, 167: 105057. doi: 10.1016/j.compag.2019.105057.
[14] ANDERSON N T, UNDERWOOD J P, RAHMAN M M, et al. Estimation of fruit load in mango orchards: Tree sampling considerations and use of machine vision and satellite imagery[J]. Precision Agriculture, 2018, 20(4): 823-839.
[15] ROUT R, PARIDA P. A review on leaf disease detection using computer vision approach[J]. International Conference on Innovation in Modern Science and Technology, 2019: 863-871. doi: 10.1007/978-3-030-42363-6_99.
[16] PATRICIO D I, RIEDER R. Computer vision and artificial intelligence in precision agriculture for grain crops: A systematic review[J]. Computers and Electronics in Agriculture, 2018, 153: 69-81. doi: 10.1016/j.compag.2018.08.001.
[17] 罗陆锋, 邹湘军, 王成琳, 等. 基于轮廓分析的双串叠贴葡萄目标识别方法[J]. 农业机械学报, 2017, 48(6): 15-22. doi: 10.6041/j.issn.1000-1298.2017.06.002. [18] HOČEVAR M, ŠIROK B, GODEŠA T, et al. Flowering estimation in apple orchards by image analysis[J]. Precision Agriculture, 2014, 15(4): 466-478. doi: 10.1007/s11119-013-9341-6
[19] GEORGESCU M I, IONESCU R T. Teacher-student training and triplet loss for facial expression recognition under occlusion[C]//2020 25th International Conference on Pattern Recognition (ICPR). Milan, Italy: IEEE, 2021: 2288-2295. doi: 10.1109/ICPR48806.2021.9412493
[20] REDMON J, FARHADI A. YOLOv3: An incremental improvement[EB/OL]. 2018 [2020-07-20].http://preddie.commedia/files/YOLOv3.pdf.
[21] ALEXE B, DESELAERS T, FERRARI V. Measuring the objectness of image windows[J]. IEEE Transactions on Software Engineering, 2012, 34(11): 2189-2202.
[22] LIN P, CHEN Y M. Detection of strawberry flowers in outdoor field by deep neural network[C]// 2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC). Chongqing: IEEE, 2018: 482-486. doi: 10.1109/ICIVC.2018.8492793.
[23] TIAN Y N, YANG G D, WANG Z, et al. Instance segmentation of apple flowers using the improved mask R–CNN model[J]. Biosystems Engineering, 2020, 193: 264-278. doi: 10.1016/j.biosystemseng.2020.03.008.
[24] KOIRALA A, WALSH K B, WANG Z L, et al. Deep learning for mango (Mangifera indica) panicle stage classification[J]. Agronomy, 2020, 10(1): 143. doi: 10.3390/agronomy10010143.
[25] 陈燕, 王佳盛, 曾泽钦, 等. 大视场下荔枝采摘机器人的视觉预定位方法[J]. 农业工程学报, 2019, 35(23): 48-54. doi: 10.11975/j.issn.1002-6819.2019.23.006. [26] SONG S S, DUAN J L, YANG Z, et al. A three-dimensional reconstruction algorithm for extracting parameters of the banana pseudo-stem[J]. Optik, 2019, 185: 486-496. doi: 10.1016/j.ijleo.2019.03.125.
[27] CHEN M Y, TANG Y C, ZOU X, et al. Three-dimensional perception of orchard banana central stock enhanced by adaptive multi-vision technology[J]. Computers and Electronics in Agriculture, 2020, 174: 105508.
[28] ZOU X, YE M, LUO C, et al. Fault-tolerant design of a limited universal fruit-picking end-effector based on vision-positioning error[J]. Applied Engineering in Agriculture, 2016, 32(1): 5-18. doi: 10.1016/j.compag.2020.105508.
[29] SOVIANY P, IONESCU R T. Optimizing the trade-off between single-stage and two-stage deep object detectors using image difficulty prediction[C]//2018 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC). Timisoara, Romania: IEEE, 2018: 209-214. doi: 10.1109/SYNASC.2018.00041.
计量
- 文章访问数: 529
- HTML全文浏览量: 11
- PDF下载量: 785
