A YOLOv5s algorithm based on BiFPN and Triplet attention mechanism for identifing defective apple
-
摘要:目的
为了充分利用上下文信息和融合多尺度特征，提出基于BiFPN和Triplet注意力机制的YOLOv5s (BTF-YOLOv5s)缺陷苹果识别算法。
方法首先，加权双向特征金字塔网络(BiFPN)引入额外的权重学习不同输入特征的重要性，模型通过自顶向下和自底向上的双向路径，实现多尺度特征的反复融合，提高多尺度的检测能力；其次，将Triplet注意力机制应用到Neck层以增强模型对目标之间的关联和上下文信息的表示能力，使模型更专注于苹果特征的学习；最后，采用Focal-CIoU损失函数调整损失权重，使模型更关注缺陷苹果的识别，提升模型的感知能力。通过消融试验对比不同损失函数，改变注意力机制在YOLOv5s结构中的位置，并与主流算法进行比较。
结果BTF-YOLOv5s在初始YOLOv5s模型基础上，准确率、召回率和mAP方面分别提高了5.7、2.2和3.5个百分点，模型内存使用量为14.7 MB；平均精度比SSD、YOLOv3、YOLOv4、YOLOv5s、YOLOv7、YOLOv8n、YOLOv8s和YOLOv9的分别提升了5.7、3.5、13.3、3.5、2.9、2.6、2.8和0.3个百分点。
结论模型在缺陷苹果识别中表现出显著的优越性，为采摘机器人在采摘过程中实现优质苹果与缺陷苹果的自动分拣提供了一定的技术支持。
Abstract:ObjectiveIn order to make full use of context information and integrate multi-scale features, a YOLOv5s algorithm based on BiFPN and Triplet attention mechanism (BTF-YOLOv5s) for identifing defective apple was proposed.
MethodFirstly, the additional weights were introduced to the weighted bidirectional feature pyramid network ( BiFPN ) to learn the importance of different input features. The model realized the repeated fusion of multi-scale features through the top-down and bottom-up bidirectional paths, and improved the multi-scale detection ability. Secondly, the Triplet attention mechanism was applied to the Neck layer to enhance the model's ability to represent the correlation between target and contextual information, so that the model could focus more on the learning of apple features. Finally, the Focal-CIoU loss function was used to adjust the loss weight, so that the model payed more attention to defective apple recognition, and improved the perception ability of the model. Different loss functions were compared through ablation experiments. The position of attention mechanism in YOLOv5 structure was changed, and compared with the mainstream algorithms.
ResultOn the basis of the initial YOLOv5s model, BTF-YOLOv5s improved the accuracy, recall and mAP by 5.7, 2.2 and 3.5 percentage points respectively, and the memory usage of the model was 14.7 MB. The average accuracy of BTF-YOLOv5s was 5.7, 3.5, 13.3, 3.5, 2.9, 2.6, 2.8 and 0.3 percentage points higher than those of SSD, YOLOv3, YOLOv4, YOLOv5s, YOLOv7, YOLOv8n, YOLOv8s and YOLOv9, respectively.
ConclusionThe model of BTF-YOLOv5s shows significant superiority in identifing defective apples, which provides certain technical support for the picking robot to realize the automatic sorting of high-quality apples and defective apples in the picking process.
-
Keywords:
- YOLOv5s /
- Defective apple /
- Attention mechanism /
- Loss function /
- Object detection /
- Picking robot
-
表 1 Triplet注意力机制不同插入位置的模型试验结果对比
Table 1 Result comparison of Triplet attention mechanism with different insertion position
% 模型1)Model P R mAP F1 YOLOv5s 75.4 83.6 86.5 79.29 A+ CIoU+Focal(9) 77.3 81 85.6 79.11 A+CIoU+Focal(9+25) 80.8 79.9 86.2 80.35 A+CIoU+Focal 75.9 85.2 87.1 80.28 A+CIoU+Focal(24) 81.1 85.8 90.0 83.38 1) A：YOLOv5s-BiFPN-Triplet，括号内数字表示Triplet插入位置。
1) The number in parentheses indicated the insertion position of Triplet.
下载: 导出CSV
表 2 BiFPN添加不同注意力机制的模型试验结果对比
Table 2 Result comparison of BiFPN with different attention mechanism
% 模型Model P R mAP BiFPN+SE 76.1 83.1 87.0 BiFPN+CBAM 77.5 81.5 87.6 BiFPN+CA 78.6 81.3 88.1 BiFPN+Triplet 79.3 83.2 88.4
下载: 导出CSV
表 3 消融试验结果
Table 3 The result of ablation test
% BiFPN Triplet Focal-CIoU P R mAP 75.4 83.6 86.5 √ 80.3 84.3 88.0 √ 76.4 86.1 89.3 √ 78.4 83.0 89.8 √ √ 79.3 83.2 88.4 √ √ 77.5 84.5 89.6 √ √ 80.1 84.0 88.9 √ √ √ 81.1 85.8 90.0
下载: 导出CSV
表 4 Focal-CIoU与其他损失函数对比
Table 4 Comparison of Focal-CIoU with other loss functions
% 模型1)Model P R mAP F1 A 79.3 83.2 88.4 81.20 A+ DIoU 79.1 84.8 89.2 81.85 A+ DIoU+Focal 79.4 84.3 89.6 81.78 A+ SIoU 80.1 85.6 89.8 82.76 A+ SIoU+Focal 77.9 85.3 88.5 81.43 A+ EIoU 78.7 84.3 88.7 81.40 A+ EIoU+Focal 80.7 80.0 88.1 80.35 A+ WIoU 81.1 79.4 88.2 80.24 A+CIoU+Focal 81.1 85.8 90.0 83.38 1) A：YOLOv5s-BiFPN-Triplet.
下载: 导出CSV
表 5 BTF-YOLOv5与其他模型的对比
Table 5 Comparison of BTF-YOLOv5 with other models
模型
ModelP/% R/% mAP/% 模型大小/MB
Model sizeF1/% 参数
ParameterGFLOPs1) SSD 76.3 81.7 84.3 95.5 78.91 2.49×107 31.4 YOLOv3 71.8 83.8 86.5 123.5 77.34 3.30×107 78.1 YOLOv4 84.2 65.6 76.6 256.3 73.75 6.50×107 142.3 YOLOv5s 75.4 83.6 86.5 14.4 79.29 7.03×106 16.0 YOLOv7 79.6 83.1 87.1 142.1 81.31 3.72×107 105.1 YOLOv8n 81.7 76.6 87.4 6.2 79.07 3.01×106 8.2 YOLOv8s 73.6 83.0 87.2 22.5 78.02 1.11×107 28.6 YOLOv9 86.0 78.8 89.7 102.8 82.24 5.10×107 238.9 BTF-YOLOv5s 81.1 85.8 90.0 14.7 83.38 7.17×106 16.7 1) GFLOPs：每秒10亿次的浮点运算数。
1) GFLOPs: Giga floating-point operations per second.
下载: 导出CSV
-
[1] 牛桂草, 宋卓展, 刘畅, 等. 中国苹果贸易竞争力评价与分析[J]. 河北农业科学, 2022, 26(3): 97-100. [2] 李大华, 孔舒, 李栋, 等. 基于改进YOLOv7的苹果表面缺陷轻量化检测算法[J]. 河南农业科学, 2024, 53(3): 141-150. [3] 李大华, 赵辉, 于晓. 基于改进谱聚类的重叠绿苹果识别方法(英文)[J]. 光谱学与光谱分析, 2019, 39(9): 2974-2981. [4] 王迎超, 张婧婧, 贾东霖, 等. 基于K-means聚类和改进MLP的苹果分级研究[J]. 河南农业科学, 2023, 52(1): 161-171. [5] 宋怡焕, 饶秀勤, 应义斌. 基于DT-CWT和LS-SVM的苹果果梗/花萼和缺陷识别[J]. 农业工程学报, 2012, 28(9): 114-118. [6] 张震, 周俊, 江自真, 等. 基于改进YOLO v7轻量化模型的自然果园环境下苹果识别方法[J]. 农业机械学报, 2024, 55(3): 231-242. doi: 10.6041/j.issn.1000-1298.2024.03.023 [7] 袁杰, 谢霖伟, 郭旭, 等. 基于改进YOLO v7的苹果叶片病害检测方法[J]. 农业机械学报, 2024, 55(11): 68-74. [8] 张莉, 王晓格, 鲍春, 等. 轻量级多任务的苹果成熟度分类模型(特邀)[J]. 激光与光电子学进展, 2024, 61(20): 141-149. [9] 闫彬, 樊攀, 王美茸, 等. 基于改进YOLOv5m的采摘机器人苹果采摘方式实时识别[J]. 农业机械学报, 2022, 53(9): 28-38. [10] 张阳婷, 黄德启, 王东伟, 等. 基于深度学习的目标检测算法研究与应用综述[J]. 计算机工程与应用, 2023, 59(18): 1-13. [11] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031
[12] HE K, GKIOXARI G, DOIIAR P, et al. Mask R-CNN [C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV). Venice: IEEE, 2017: 2961-2969.
[13] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]// Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 2999-3007.
[14] LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot MultiBox detector[C]// Computer Vision – ECCV 2016. Cham: Springer International Publishing, 2016: 21-37.
[15] LIU X D, GONG W Y, SHANG L L, et al. Remote sensing image target detection and recognition based on YOLOv5[J]. Remote Sensing, 2023, 15(18): 4459. doi: 10.3390/rs15184459
[16] LI C, LI L, JIANG H, et al. YOLOv6: A single-stage object detection framework for industrial applications[EB/OL]. arXiv: 2209.02976. (2022-09-07)[2024-06-18]. https://doi.org/10.48550/arXiv.2209.02976.
[17] WANG C Y, BOCHKOVSKIY A, LIAO H M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023: 7464-7475.
[18] TIAN L L, ZHANG H X, LIU B, et al. VMF-SSD: A novel V-space based multi-scale feature fusion SSD for apple leaf disease detection[J]. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2023, 20(3): 2016-2028. doi: 10.1109/TCBB.2022.3229114
[19] WANG D D, HE D J. Channel pruned YOLO V5s-based deep learning approach for rapid and accurate apple fruitlet detection before fruit thinning[J]. Biosystems Engineering, 2021, 210: 271-281. doi: 10.1016/j.biosystemseng.2021.08.015
[20] 化春键, 孙明春, 蒋毅, 等. 基于改进YOLOv7-tiny的多光谱苹果表层缺陷检测[J]. 激光与光电子学进展, 2024, 61(10): 236-244. [21] 朱琦, 周德强, 盛卫锋, 等. 基于DSCS-YOLO的苹果表面缺陷检测方法[J]. 南京农业大学学报, 2024, 47(3): 592-601. [22] LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 936-944.
[23] LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8759-8768.
[24] TAN M X, PANG R M, LE Q V. EfficientDet: Scalable and efficient object detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2020: 10778-10787.
[25] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[26] WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional block attention module[C]// Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[27] MISRA D, NALAMADA T, ARASANIPALAI A U, et al. Rotate to attend: Convolutional triplet attention module[C]//2021 IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa: IEEE, 2021: 3138-3147.
[28] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 318-327. doi: 10.1109/TPAMI.2018.2858826
[29] ZHENG Z H, WANG P, LIU W, et al. Distance-IoU loss: Faster and better learning for bounding box regression[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 12993-13000. doi: 10.1609/aaai.v34i07.6999
-
期刊类型引用(14)
1. 潘婷婷，康喜龙，黄霞，孟闯，潘志明，焦新安. 非洲猪瘟诊断靶标的研究进展. 微生物学通报. 2025(02): 510-521 . 
百度学术
2. 苗雅倩，杨艳歌，赵健淞，魏莹，王秀娟，袁飞，王正亮，张峰. 配方乳粉中6种致病菌双重酶促等温扩增快速检测方法的建立. 微生物学报. 2025(03): 1319-1336 . 百度学术
3. 赵健淞，杨艳歌，刘通，魏莹，李红娜，李涛，孙冬梅，袁飞. 现制直饮品中铜绿假单胞菌的ERA快速检测. 中国食品学报. 2025(04): 376-385 . 百度学术
4. 华冰洁. 非洲猪瘟的研究进展. 中国动物保健. 2024(02): 7-8+28 . 百度学术
5. 潘建良. 非洲猪瘟防控期间养猪场生物安全管理探讨. 今日畜牧兽医. 2023(02): 22-23+26 . 百度学术
6. 王帅，杨艳歌，吴占文，李红娜，李涛，孙冬梅，袁飞. 重组酶聚合酶扩增、重组酶介导等温扩增及酶促重组等温扩增技术在食源性致病菌快速检测中的研究进展. 食品科学. 2023(09): 297-305 . 百度学术
7. 林志伟，王帅，王迎春，吴占文，李涛，李红娜，杨艳歌，袁飞. 婴儿配方乳粉中食源性致病菌双重ERA快速检测方法的建立. 食品科学. 2023(18): 347-354 . 百度学术
8. 杨艳歌，王帅，李红娜，李涛，吴占文，孙冬梅，袁飞. 沙门氏菌ERA可视化快速检测方法的建立. 中国食品学报. 2023(10): 261-272 . 百度学术
9. 杨艳歌，吴占文，李涛，王帅，李红娜，孙冬梅，袁飞. GI和GII诺如病毒ERA的可视化快速检测. 华南理工大学学报(自然科学版). 2023(12): 140-151 . 百度学术
10. 王翠，许宗丽，黄溢泓，覃艳然，梁竞臻，周师师，刘针伶，黄小武，马小蓉，李志源，严斯刚. 非洲猪瘟病原学检测技术研究进展. 中国动物检疫. 2022(02): 74-81 . 百度学术
11. 张帅，刘媛，赵云环，刘莹，左玉柱，范京惠. 非洲猪瘟病毒研究进展. 中国兽医学报. 2022(12): 2569-2577 . 百度学术
12. 魏晓通，周宇航，刘扬，荣馨锐，张士新，徐亚维. 酶促恒温扩增技术快速鉴定食品中牡蛎成分. 食品科技. 2022(11): 255-261 . 百度学术
13. 倪俊芬，陈前岭，仲向前. 我国对非洲猪瘟病毒的研究进展综述. 中国猪业. 2022(06): 70-74 . 百度学术
14. 王良辉. 精准清洗消毒是防控非洲猪瘟的重要措施. 兽医导刊. 2021(23): 79-80 . 百度学术
其他类型引用(4)
