| Citation: |
XIONG Juntao, HUO Zhaowei, HUANG Qiyin, et al. Detection method of citrus in nighttime environment combined with active light source and improved YOLOv5s model[J]. Journal of South China Agricultural University, 2024, 45(1): 97-107. DOI: 10.7671/j.issn.1001-411X.202209010
|
To solve the problems of occlusion and difficult identification of small citrus in the nighttime environment and redlize the all-weather intelligent operation of picking robots.
A nighttime citrus identification method combined with active light sources was proposed in this paper. Firstly, the best illuminant color was selected by analyzing nighttime citrus images with different color features under active light sources. Then, a nighttime citrus detection model named BI-YOLOv5s was proposed with bi-directional feature pyramid network (Bi-FPN) for multi-scale cross connection and weighted feature fusion to improve the detection performance of occlusion and small citruses. The coordinate attention (CA) module with attention mechanism was introduced to further strengthen the extraction of target location information. Meanwhile, the C3TR module integrated with a Transformer structure was adopted to reduce the computing amount and better extract global information.
The precision, recall and average precision of the citrus detection using the BI-YOLOv5s on test set were 93.4%, 92.2% and 97.1%, respectively, with 3.2, 1.5 and 2.3 percent higher than the YOLOv5s, respectively. Moreover, the identification accuracy of the proposed model with an active light source for nighttime citrus was 95.3%, with 10.4 percent higher than the model in the white light environment.
The proposed method in this paper has high accuracy for the identification of occlusion and small target citrus in the nighttime environment, and it can provide technical support for nighttime visual identification of intelligent picking of fruits and vegetables.
| [1] |
王刘坤, 祁春节. 中国柑橘主产区的区域比较优势及其影响因素研究: 基于省级面板数据的实证分析[J]. 中国农业资源与区划, 2018, 39(11): 121-128.
|
| [2] |
REN D, YANG S. Intelligent automation with applications to agriculture[J]. Intelligent Automation and Soft Computing, 2016, 22(2): 227-228. doi: 10.1080/10798587.2015.1095473
|
| [3] |
KOIRALA A, WALSH K B, WANG Z, et al. Deep learning: Method overview and review of use for fruit detection and yield estimation[J]. Computers and Electronics in Agriculture, 2019, 162: 219-234. doi: 10.1016/j.compag.2019.04.017
|
| [4] |
杨长辉, 刘艳平, 王毅, 等. 自然环境下柑橘采摘机器人识别定位系统研究[J]. 农业机械学报, 2019, 50(12): 14-22. doi: 10.6041/j.issn.1000-1298.2019.12.002
|
| [5] |
XIANG R. Image segmentation for whole tomato plant recognition at night[J]. Computers and Electronics in Agriculture, 2018, 154: 434-442. doi: 10.1016/j.compag.2018.09.034
|
| [6] |
HAYASHI S, SHIGEMATSU K, YAMAMOTO S, et al. Evaluation of a strawberry-harvesting robot in a field test[J]. Biosystems Engineering, 2010, 105(2): 160-171. doi: 10.1016/j.biosystemseng.2009.09.011
|
| [7] |
AREFI A, MOTLAGH A M, MOLLAZADE K, et al. Recognition and localization of ripen tomato based on machine vision[J]. Australian Journal of Crop Science, 2011, 5(10): 1144-1149.
|
| [8] |
JI W, QIAN Z, XU B, et al. A nighttime image enhancement method based on Retinex and guided filter for object recognition of apple harvesting robot[J]. International Journal of Advanced Robotic Systems, 2018, 15(1): 1729881417753871.
|
| [9] |
PAYNE A, WALSH K, SUBEDI P, et al. Estimating mango crop yield using image analysis using fruit at ‘stone hardening’ stage and night time imaging[J]. Computers and Electronics in Agriculture, 2014, 100: 160-167. doi: 10.1016/j.compag.2013.11.011
|
| [10] |
TANG Y, CHEN M, WANG C, et al. Recognition and localization methods for vision-based fruit picking robots: A review[J]. Frontiers in Plant Science, 2020, 11: 510. doi: 10.3389/fpls.2020.00510.
|
| [11] |
XIONG J, LIN R, LIU Z, et al. The recognition of litchi clusters and the calculation of picking point in a nocturnal natural environment[J]. Biosystems Engineering, 2018, 166: 44-57. doi: 10.1016/j.biosystemseng.2017.11.005
|
| [12] |
FU L, WANG B, CUI Y, et al. Kiwifruit recognition at nighttime using artificial lighting based on machine vision[J]. International Journal of Agricultural and Biological Engineering, 2015, 8(4): 52-59.
|
| [13] |
项荣, 段鹏飞. 番茄采摘机器人夜间照明系统设计与试验[J]. 农业机械学报, 2016, 47(7): 8-14. doi: 10.6041/j.issn.1000-1298.2016.07.002
|
| [14] |
赵德安, 刘晓洋, 陈玉, 等. 苹果采摘机器人夜间识别方法[J]. 农业机械学报, 2015, 46(3): 15-22. doi: 10.6041/j.issn.1000-1298.2015.03.003
|
| [15] |
LIANG C, XIONG J, ZHENG Z, et al. A visual detection method for nighttime litchi fruits and fruiting stems[J]. Computers and Electronics in Agriculture, 2020, 169: 105192. doi: 10.1016/j.compag.2019.105192.
|
| [16] |
王辉, 张帆, 刘晓凤, 等. 基于DarkNet-53和YOLOv3的水果图像识别[J]. 东北师大学报(自然科学版), 2020, 52(4): 60-65.
|
| [17] |
熊俊涛, 郑镇辉, 梁嘉恩, 等. 基于改进YOLO v3网络的夜间环境柑橘识别方法[J]. 农业机械学报, 2020, 51(4): 199-206. doi: 10.6041/j.issn.1000-1298.2020.04.023
|
| [18] |
吕石磊, 卢思华, 李震, 等. 基于改进YOLOv3-LITE轻量级神经网络的柑橘识别方法[J]. 农业工程学报, 2019, 35(17): 205-214. doi: 10.11975/j.issn.1002-6819.2019.17.025
|
| [19] |
郑太雄, 江明哲, 冯明驰. 基于视觉的采摘机器人目标识别与定位方法研究综述[J]. 仪器仪表学报, 2021, 42(9): 28-51.
|
| [20] |
CHEN J, WU J, WANG Z, et al. Detecting ripe fruits under natural occlusion and illumination conditions[J]. Computers and Electronics in Agriculture, 2021, 190: 106450. doi: 10.1016/j.compag.2021.106450.
|
| [21] |
熊俊涛, 林睿, 刘振, 等. 夜间自然环境下荔枝采摘机器人别技术[J]. 农业机械学报, 2017, 48(11): 28-34. doi: 10.6041/j.issn.1000-1298.2017.11.004
|
| [22] |
LIU X, ZHAO D, JIA W, et al. A method of segmenting apples at night based on color and position information[J]. Computers and Electronics in Agriculture, 2016, 122: 118-123. doi: 10.1016/j.compag.2016.01.023
|
| [23] |
贾伟宽, 赵德安, 阮承治, 等. 苹果采摘机器人夜间图像降噪算法[J]. 农业工程学报, 2015, 31(10): 219-226. doi: 10.11975/j.issn.1002-6819.2015.10.029
|
| [24] |
赵桂兰, 姚其. 蔬果照明光源与物体的照明光谱反射关系研究[J]. 灯与照明, 2019, 43(2): 1-5. doi: 10.3969/j.issn.1008-5521.2019.02.002
|
| [25] |
HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design[EB/OL]. arXiv: 2103.02907v1 (2021-03-04) [2022-09-09]. https://arxiv.org/pdf/2103.02907.pdf.
|
| [26] |
TAN M, PANG R, LE Q. EfficientDet: Scalable and efficient object detection[C]//Proceedings of IEEE/CFV Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2020: 10781-10790.
|
| [27] |
LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018: 8759-8768.
|
| [28] |
ZHU X, LYU S, WANG X, et al. TPH-YOLOv5: Improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Virtual Conference: IEEE, 2021: 2778-2788.
|
| [29] |
GLENN J. YOLOv5[CP/OL]. (2021-01-15) [2022-03-12]. https://github.com/ultralytics/yoloV5.
|
| 1. |
陈灿，陈维林，丁浩，陈兰，张跟喜，谢恺舟，戴国俊，王金玉，张涛. 白羽王鸽胚胎期胸肌肌纤维发育规律及相关基因表达分析. 中国畜牧杂志. 2023(07): 99-104 .
| |
| 2. |
陈秋阳，朱康平，罗毅，沈林園，朱砺，甘麦邻. 猪原代骨骼肌卫星细胞的快速分离、培养和诱导分化研究. 中国畜牧杂志. 2023(08): 118-123 .
| |
| 3. |
张力，许加龙，黄锦钰，许子月，雷昕诺，卢会鹏，朱睿，孙伟翔，曹海月，王安平，朱善元. 鹅骨骼肌卫星细胞的分离培养与鉴定. 畜牧兽医学报. 2023(10): 4186-4195 .
| |
| 4. |
王燕星，张雨时，姬海港，刘阳，牛玉芳，韩瑞丽，刘小军，田亚东，康相涛，李转见. 鸡骨骼肌卫星细胞系的建立及分析. 畜牧兽医学报. 2023(12): 4972-4981 .
| |
| 5. |
戴巍，宋瑞龙，张远浩，邹辉，顾建红，袁燕，卞建春，刘学忠. 鸡骨骼肌卫星细胞的分离培养与鉴定. 畜牧兽医学报. 2021(03): 676-682 .
| |
| 6. |
马思佳，刘媛，汪序忠，李亭亭，段星，杨松柏，宋丹，李向臣. 褪黑素对脂多糖刺激的滩羊骨骼肌卫星细胞炎性因子表达的影响. 中国畜牧兽医. 2021(09): 3378-3386 .
| |
| 7. |
徐小春，赵瑞，陈文娟，马文平，马森. 滩羊骨骼肌卫星细胞体外培养及成肌和成脂诱导分化研究. 家畜生态学报. 2020(12): 32-39 .
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: