Research on fast recognition of banana multi-target features by visual robot in complex environment

WU Fengyun; YE Yaxin; CHEN Siyu; AI Puye; ZOU Xiangjun; DUAN Jieli

doi:10.7671/j.issn.1001-411X.202108009

Journal of South China Agricultural University > 2022 > 43(2): 96-104. > DOI: 10.7671/j.issn.1001-411X.202108009

WU Fengyun, YE Yaxin, CHEN Siyu, et al. Research on fast recognition of banana multi-target features by visual robot in complex environment[J]. Journal of South China Agricultural University, 2022, 43(2): 96-104. DOI: 10.7671/j.issn.1001-411X.202108009

Citation:

PDF (1165 KB)

Research on fast recognition of banana multi-target features by visual robot in complex environment

College of Engineering, South China Agricultural University, Guangzhou 510642, China

More Information

Received Date: August 09, 2021
Available Online: May 17, 2023

Graphical Abstract

Abstract

Abstract

Objective
Aiming at fast recognition of multi-feature variable target by robot vision in field environment, and considering the problem that the accuracy of target is affected by leaves, shade and light, a fast recognition method of multi-feature target was proposed.

Method
A multi-scale feature extraction and classification model was proposed for three targets including banana fruit, fruit axis and flower bud. New target candidate box parameters were designed using clustering algorithm. The network structure parameters of YOLOv3 model were optimized and the YOLO-Banana model was proposed. In order to balance the speed and accuracy, YOLO-Banana and Faster R-CNN were used to conduct experiments on banana multi-targets with varying sizes. The effects of algorithms on recognition accuracy and speed were analyzed.

Result
The average accuracies of YOLO-Banana and Faster R-CNN algorithms on banana fruit, fruit axis and flower bud were 91.03% and 95.16% respectively, average recognition time per photo was 0.237 and 0.434 s respectively. Therefore the accuracies of two algorithms were both above 90%. YOLO-Banana had relatively 1.83 times faster speed than Faster R-CNN, better satisfying the requirement of real-time operation.

Conclusion
In the wild environment, utilization of YOLO-Banana model for banana multi-target recognition can meet the speed and accuracy requirements for visual recognition by the bud-breaking robots.
- Deep learning,
- Flower bud,
- Fruit axis,
- Multi-feature,
- Multi-target recognition,
- Fast classification

FullText(HTML)

References (29)

References

[1]	YIN Y H, LI H F, FU W. Faster-YOLO: An accurate and faster object detection method[J]. Digital Signal Processing, 2020, 102: 102756. doi: 10.1016/j.dsp.2020.102756.
[2]	何银水, 余卓骅, 李健, 等. 基于视觉特征的厚板机器人焊接焊缝轮廓的有效提取[J]. 机械工程学报, 2019, 55(17): 56-60.
[3]	訾斌, 尹泽强, 李永昌, 等. 基于YOLO模型的柔索并联机器人移动构件快速定位方法[J]. 机械工程学报, 2019, 55(3): 64-72.
[4]	孙国鹏, 郝向阳, 张振杰, 等. 多特征判断的合作目标识别方法[J]. 系统仿真学报, 2018, 30(6): 2377-2383.
[5]	薛培林, 吴愿, 殷国栋, 等. 基于信息融合的城市自主车辆实时目标识别[J]. 机械工程学报, 2020, 56(12): 165-173.
[6]	张青, 邹湘军, 林桂潮, 等. 草莓重量和形状图像特征提取与在线分级方法[J]. 系统仿真学报, 2019, 31(1): 7-15.
[7]	TANG Y C, LI L J, WANG C L, et al. Real-time detection of surface deformation and strain in recycled aggregate concrete-filled steel tubular columns via four-ocular vision[J]. Robotics and Computer-Integrated Manufacturing, 2019, 59: 36-46. doi: 10.1016/j.rcim.2019.03.001
[8]	DAI Z, SONG H S, LIANG H X, et al. Traffic parameter estimation and control system based on machine vision[J]. Journal of Ambient Intelligence and Humanized Computing, 2020: 1-13. doi: 10.1007/s12652-020-02052-5.
[9]	MIAN A S, BENNAMOUN M, OWENS R. Three-dimensional model-based object recognition and segmentation in cluttered scenes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(10): 1584-1601. doi: 10.1109/TPAMI.2006.213
[10]	WEI X Q, JIA K, LAN J H, et al. Automatic method of fruit object extraction under complex agricultural background for vision system of fruit picking robot[J]. Optik, 2014, 125(19): 5684-5689. doi: 10.1016/j.ijleo.2014.07.001.
[11]	GONGAL A, AMATYA S, KARKEE M, et al. Sensors and systems for fruit detection and localization: A review[J]. Computers and Electronics in Agriculture, 2015, 116: 8-19. doi: 10.1016/j.compag.2015.05.021.
[12]	TANG Y C, CHEN M Y, WANG C L, et al. Recognition and localization methods for vision-based fruit picking robots: A review[J]. Frontiers in Plant Science, 2020, 11: 510. doi: 10.3389/fpls.2020.00510.
[13]	FU L H, DUAN J L, ZOU X J, et al. Banana detection based on color and texture features in the natural environment[J]. Computers and Electronics in Agriculture, 2019, 167: 105057. doi: 10.1016/j.compag.2019.105057.
[14]	ANDERSON N T, UNDERWOOD J P, RAHMAN M M, et al. Estimation of fruit load in mango orchards: Tree sampling considerations and use of machine vision and satellite imagery[J]. Precision Agriculture, 2018, 20(4): 823-839.
[15]	ROUT R, PARIDA P. A review on leaf disease detection using computer vision approach[J]. International Conference on Innovation in Modern Science and Technology, 2019: 863-871. doi: 10.1007/978-3-030-42363-6_99.
[16]	PATRICIO D I, RIEDER R. Computer vision and artificial intelligence in precision agriculture for grain crops: A systematic review[J]. Computers and Electronics in Agriculture, 2018, 153: 69-81. doi: 10.1016/j.compag.2018.08.001.
[17]	罗陆锋, 邹湘军, 王成琳, 等. 基于轮廓分析的双串叠贴葡萄目标识别方法[J]. 农业机械学报, 2017, 48(6): 15-22. doi: 10.6041/j.issn.1000-1298.2017.06.002.
[18]	HOČEVAR M, ŠIROK B, GODEŠA T, et al. Flowering estimation in apple orchards by image analysis[J]. Precision Agriculture, 2014, 15(4): 466-478. doi: 10.1007/s11119-013-9341-6
[19]	GEORGESCU M I, IONESCU R T. Teacher-student training and triplet loss for facial expression recognition under occlusion[C]//2020 25th International Conference on Pattern Recognition (ICPR). Milan, Italy: IEEE, 2021: 2288-2295. doi: 10.1109/ICPR48806.2021.9412493
[20]	REDMON J, FARHADI A. YOLOv3: An incremental improvement[EB/OL]. 2018 [2020-07-20].http://preddie.commedia/files/YOLOv3.pdf.
[21]	ALEXE B, DESELAERS T, FERRARI V. Measuring the objectness of image windows[J]. IEEE Transactions on Software Engineering, 2012, 34(11): 2189-2202.
[22]	LIN P, CHEN Y M. Detection of strawberry flowers in outdoor field by deep neural network[C]// 2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC). Chongqing: IEEE, 2018: 482-486. doi: 10.1109/ICIVC.2018.8492793.
[23]	TIAN Y N, YANG G D, WANG Z, et al. Instance segmentation of apple flowers using the improved mask R–CNN model[J]. Biosystems Engineering, 2020, 193: 264-278. doi: 10.1016/j.biosystemseng.2020.03.008.
[24]	KOIRALA A, WALSH K B, WANG Z L, et al. Deep learning for mango (Mangifera indica) panicle stage classification[J]. Agronomy, 2020, 10(1): 143. doi: 10.3390/agronomy10010143.
[25]	陈燕, 王佳盛, 曾泽钦, 等. 大视场下荔枝采摘机器人的视觉预定位方法[J]. 农业工程学报, 2019, 35(23): 48-54. doi: 10.11975/j.issn.1002-6819.2019.23.006.
[26]	SONG S S, DUAN J L, YANG Z, et al. A three-dimensional reconstruction algorithm for extracting parameters of the banana pseudo-stem[J]. Optik, 2019, 185: 486-496. doi: 10.1016/j.ijleo.2019.03.125.
[27]	CHEN M Y, TANG Y C, ZOU X, et al. Three-dimensional perception of orchard banana central stock enhanced by adaptive multi-vision technology[J]. Computers and Electronics in Agriculture, 2020, 174: 105508.
[28]	ZOU X, YE M, LUO C, et al. Fault-tolerant design of a limited universal fruit-picking end-effector based on vision-positioning error[J]. Applied Engineering in Agriculture, 2016, 32(1): 5-18. doi: 10.1016/j.compag.2020.105508.
[29]	SOVIANY P, IONESCU R T. Optimizing the trade-off between single-stage and two-stage deep object detectors using image difficulty prediction[C]//2018 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC). Timisoara, Romania: IEEE, 2018: 209-214. doi: 10.1109/SYNASC.2018.00041.

[1]	CHEN Shengde, CHEN Yigang, XU Xiaojie, LIU Junyu, GUO Jianzhou, HU Shiyun, LAN Yubin. Monitoring of corn leaf area index based on multispectral remote sensing of UAV[J]. Journal of South China Agricultural University, 2024, 45(4): 608-617. DOI: 10.7671/j.issn.1001-411X.202310025
[2]	YUE Xuejun, SONG Qingkui, LI Zhiqing, ZHENG Jianyu, XIAO Jiayi, ZENG Fanguo. Research status and prospect of crop information monitoring technology in field[J]. Journal of South China Agricultural University, 2023, 44(1): 43-56. DOI: 10.7671/j.issn.1001-411X.202209042
[3]	DILIXIATI Yimamu, ZHOU Jianping, XU Yan, FAN Xiangpeng, YALIKUN Shawuti. Cotton pest monitoring based on Logistic algorithm and remote sensing image[J]. Journal of South China Agricultural University, 2022, 43(2): 87-95. DOI: 10.7671/j.issn.1001-411X.202106004
[4]	DING Youchun, CHEN Liyuan, WANG Denghui, LIU Xiaodong, XU Chunbao, WANG Kaiyang. Design and test of monitoring system for rapeseed sowing quality[J]. Journal of South China Agricultural University, 2021, 42(6): 43-51. DOI: 10.7671/j.issn.1001-411X.202107029
[5]	LIU Chuanling, CHEN Ming, CHI Tao. Design and application of aquaculture monitoring system based on LoRa wireless communication[J]. Journal of South China Agricultural University, 2020, 41(6): 154-160. DOI: 10.7671/j.issn.1001-411X.202006043
[6]	XU Peiquan, SHEN Mingxia, LIU Longshen, HE Canlong, KANG Jian, TAO Yuandong. Design and implementation of piggery environmental monitoring system based on ECS and WSN[J]. Journal of South China Agricultural University, 2018, 39(1): 112-119. DOI: 10.7671/j.issn.1001-411X.2018.01.018
[7]	YUE Xuejun, WANG Yefu, LIU Yongxin, XU Xing, CHEN Shurong, CHEN Yixi, HOU Miancong, YAN Yingwei, QUAN Dongping, CHEN Zhuliang. Orchard environmental monitoring system based on GPRS and ZigBee[J]. Journal of South China Agricultural University, 2014, 35(4): 109-113. DOI: 10.7671/j.issn.1001-411X.2014.04.020
[8]	SANG Song, WANG Peidan, CAO Yuke, ZHONG Guohua. A study on Margarya melanioides and Cipangopaludina chinensis as the indicator organisms to monitor pesticide residues in a tea garden environment[J]. Journal of South China Agricultural University, 2014, 35(3): 58-62. DOI: 10.7671/j.issn.1001-411X.2014.03.011
[9]	XU Xing, YUE Xuejun, LIN Tao. The Design of Wireless Monitoring System of Water Environment Based on the ZigBee Network[J]. Journal of South China Agricultural University, 2013, 34(4): 593-597. DOI: 10.7671/j.issn.1001-411X.2013.04.027
[10]	WANG Jian-wu,XIAO Hong-sheng,ZHANG Li-ming. Application of GPS to Monitoring Dynamics of Reclaiming Coastal Land[J]. Journal of South China Agricultural University, 2000, (2): 1-4. DOI: 10.7671/j.issn.1001-411X.2000.02.001

Cited By

Cited by

Periodical cited type(5)

1.	苏红梅，韩丽萍，刘俊华. 数字赋能内蒙古牛羊肉供应链绿色转型研究. 内蒙古社会科学. 2025(01): 205-212 .
2.	梁金鑫，苑志宇，罗新惠，陈耿，吴毓瑾，王春昕. 家畜高通量体尺测量及体重预估技术研究进展. 中国畜牧兽医. 2025(02): 749-758 .
3.	余仕亮. 云南省景东县肉牛疾病防治措施. 现代畜牧科技. 2025(04): 111-113 .
4.	徐君鹏，时磊，王宇，杨文强，杨秋亚. 基于PLC的生猪养殖智能化环境监控及云平台系统设计. 河南科技学院学报(自然科学版). 2025(03): 44-53 .
5.	赵晓霞，袁洪波，程曼. 利用深度学习实现羊只多目标跟踪. 河北农业大学学报. 2024(06): 24-31 .

Other cited types(0)

Get Citation

PDF

XML

Article views (529) PDF downloads (785) Cited by(5)

Turn off MathJax

Article Contents

Abstract

References

Research on fast recognition of banana multi-target features by visual robot in complex environment