Fine-grained tomato disease recognition based on attention residual mechanism
-
摘要:目的
解决温室环境下细粒度番茄病害识别方法不足问题。
方法以早、晚期5种番茄病害叶片为研究对象,提出一种基于注意力与残差思想相结合的新型卷积神经网络模型ARNet。通过引入多层注意力模块,层次化抽取病害分类信息,解决早期病害部位分散、特征难以提取难题;为避免网络训练出现退化现象,构建残差模块有效融合高低阶特征,同时引入数据扩充技术以防止模型过拟合。
结果对44 295张早、晚期病害叶片数据集进行模型训练与测试的结果表明,与VGG16等现有模型相比,ARNet具有更好的分类表现,其平均识别准确率达到88.2%,显著高于其他模型。ARNet对早期病害识别准确率明显优于晚期病害,验证了注意力机制在提取细微区域特征上的有效性,且在训练过程中未发生过度抖动的状况。
结论本文提出的模型具有较强鲁棒性和较高稳定性,在实际应用中可为细粒度番茄病害智能诊断提供参考。
Abstract:ObjectiveTo solve the insufficient identification of fine-grained tomato diseases in greenhouse.
MethodTaking tomato leaves with five early or late diseases as research objects, we proposed a new convolutional neural network model ARNet based on the combination of attention and residual thought. A multi-layered attention module was introduced to solve the problem of early disease location dispersion and the difficulty of feature extraction by extracting hierarchically disease classification information. In order to avoid the degradation of network training, we constructed a residual module to effectively integrate high- and low-order features. Meantime, we introduced the data expansion technology to prevent model over-fitting.
ResultModel training and testing results of early and late disease leaf datasets with 44 295 pictures showed that ARNet has better classification performance with an average recognition accuracy of 88.2%, which was significantly higher than those of other existing models. In addition, the identification accuracy of ARNet for early disease was significantly better than that for late disease, which verified the effectiveness of attention mechanism in extracting fine region features, and there was no excessive jitter during training process.
ConclusionThis model proposed in this paper has strong robustness and high stability, and can provide a reference for intelligent diagnosis of fine-grained tomato diseases in practical application.
-
Keywords:
- fine-grained /
- attention mechanism /
- residual network /
- convolutional network /
- tomato leaf /
- disease recognition
-
-
图 3 番茄病害初始数据集分布
1:早期白粉病;2:晚期白粉病;3:早期早疫病;4:晚期早疫病;5:早期叶霉病;6:晚期叶霉病;7:早期斑枯病;8:晚期斑枯病;9:早期黄化曲叶病;10:晚期黄化曲叶病
Figure 3. The distribution of initial dataset for tomato disease
1: Early powdery mildew; 2: Later powdery mildew; 3: Early early blight; 4: Later early blight; 5: Early leaf frost disease; 6: Later leaf frost disease; 7: Early spot blight; 8: Later spot blight; 9: Early yellow flower curly leaf disease; 10: Later yellow flower curly leaf disease
图 7 3种模型测试集混淆矩阵
行号1~10分别表示早期白粉病、晚期白粉病、早期早疫病、晚期早疫病、早期叶霜病、晚期叶霜病、早期斑枯病、晚期斑枯病、早期黄化曲叶病以及晚期黄化曲叶病,列号1~10类别与其对应对应编号的行号类别一致
Figure 7. The confusion matrixes of test sets of three models
Line numbers 1 to 10 indicate early powdery mildew, later powdery mildew, early early blight, later early blight, early leaf frost disease, later leaf frost disease, early spot blight, later spot blight, early yellow flower curl leaf disease, and in the case of later yellow flower curl leaf disease, the column numbers 1 to 10 are consistent with the row number category corresponding to the corresponding number
表 1 不同模型对番茄病害的分类准确率1)
Table 1 Classification accuracy of tomato diseases by different models
病害 Disease 时期 Stage VGG16 InceptionV3 Xception MobileNetV2 ResNet34 ARNet 白粉病 Powdery mildew 早期 Early 0.176 0.191 0.197 0.521 0.521 0.638 晚期 Late $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.996}}$ 0.978 0.972 0.939 0.967 0.967 早疫病 Early blight 早期 Early 0.647 0.571 0.541 0.797 0.872 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.940}}$ 晚期 Late $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.909}}$ 0.683 0.706 0.794 0.833 0.865 叶霜病 Leaf frost disease 早期 Early 0.734 0.814 0.824 0.830 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.920}}$ $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.920}}$ 晚期 Late 0.773 0.432 0.492 0.589 0.627 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.843}}$ 斑枯病 Spot blight 早期 Early 0.573 0.300 0.496 0.712 0.838 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.888}}$ 晚期 Late 0.938 0.913 0.825 0.877 0.931 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.948}}$ 黄化曲叶病
Yellow flower Curly leaf disease早期 Early 0.715 0.693 0.713 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.830}}$ $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.830}}$ 0.826 晚期 Late $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.911}}$ 0.868 0.855 0.804 0.856 0.887 平均值 Average 0.817 0.753 0.758 0.807 0.850 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.882}}$ 1)加框数据表示对应列模型在对应行病害中的最佳准确率
1)Framed data indicated the best accuracy of corresponding column model in the corresponding row disease -
[1] BAI X, LI X, FU Z, et al. A fuzzy clustering segmentation method based on neighborhood grayscale information for defining cucumber leaf spot disease images[J]. Comput Electron Agric, 2017, 136: 157-165. doi: 10.1016/j.compag.2017.03.004
[2] 王翔宇, 温皓杰, 李鑫星, 等. 农业主要病害检测与预警技术研究进展分析[J]. 农业机械学报, 2016, 47(9): 266-277. doi: 10.6041/j.issn.1000-1298.2016.09.037 [3] 龙满生, 欧阳春娟, 刘欢, 等. 基于卷积神经网络与迁移学习的油茶病害图像识别[J]. 农业工程学报, 2018, 34(18): 194-201. doi: 10.11975/j.issn.1002-6819.2018.18.024 [4] MOKHTAR U, ALI M A S, HASSENIAN A E, et al. Tomato leaves diseases detection approach based on support vector machines[C]//IEEE. Computer Engineering Conference (ICENCO). Egypt: IEEE, 2015: 246-250.
[5] XIE C, SHAO Y, LI X, et al. Detection of early blight and late blight diseases on tomato leaves using hyperspectral imaging[J]. Sci Rep, 2015, 5: 16564. doi: 10.1038/srep16564
[6] 柴阿丽, 李宝聚, 石延霞, 等. 基于计算机视觉技术的番茄叶部病害识别[J]. 园艺学报, 2010, 37(9): 1423-1430. [7] AMARA J, BOUAZIZ B, ALGERGAWY A. A deep learning based approach for banana leaf diseases classification[M]// MITSCHANG B. Lecture Notes in Informatics (LNI). Bonn : Gesellschaft Für Informatik, 2017: 79-88.
[8] 马浚诚, 杜克明, 郑飞翔, 等. 基于卷积神经网络的温室黄瓜病害识别系统[J]. 农业工程学报, 2018, 34(12): 186-192. doi: 10.11975/j.issn.1002-6819.2018.12.022 [9] DURMUS H, GUNES E O, KIRCI M. Disease detection on the leaves of the tomato plants by using deep learning[C]//IEEE. 2017 6th International Conference on Agro-Geoinformatics. Fairfax VA: IEEE, 2017: 1-5.
[10] FUENTES A, YOON S, KIM S C, et al. A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition[J]. Sensors, 2017, 17(9): 2022. doi: 10.3390/s17092022
[11] BRAHIMI M, BOUKHALFA K, MOUSSAOUI A. Deep learning for tomato diseases: Classification and symptoms visualization[J]. Appl Artif Intell, 2017, 31(4): 299-315.
[12] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Curran Associates. Advances in Neural Information Processing Systems(NIPS). New York: Curran Associates, 2017: 5998-6008.
[13] LETARTE G, PARADIS F, GIGUERE P, et al. Importance of self-attention for sentiment analysis[C]// Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP. Brussels: Association for Computational Linguistics. 2018: 267-275.
[14] YU C, WANG J, PENG C, et al. BiSeNet: Bilateral segmentation network for real-time semantic segmentation [C]//Springer. Proceedings of the European Conference on Computer Vision (ECCV). Munich: Springer, 2018: 325-341.
[15] WANG F, JIANG M, QIAN C, et al. Residual attention network for image classification[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 3156-3164.
[16] FU J, ZHENG H, MEI T. Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 4476-4484.
[17] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
[18] HE K, ZHANG X, REN S, et al. Identity mappings in deep residual networks[C]//Springer. Proceedings of the European Conference on Computer Vision (ECCV). Amsterdam : Springer, 2016: 630-645.
[19] ZHANG K, SUN M, HAN T X, et al. Residual networks of residual networks: Multilevel residual networks[J]. IEEE T Circ Syst Vid, 2018, 28(6): 1303-1314. doi: 10.1109/TCSVT.2017.2654543
[20] 孙俊, 谭文军, 毛罕平, 等. 基于改进卷积神经网络的多种植物叶片病害识别[J]. 农业工程学报, 2017, 33(19): 209-215. doi: 10.11975/j.issn.1002-6819.2017.19.027 [21] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 2818-2826.
[22] CHOLLET F. Xception: Deep Learning with Depthwise Separable Convolutions[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 1800-1807.
[23] SANDLER M, HOWARD A, ZHU M, et al. MobileNetV2: Inverted residuals and linear bottlenecks[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 4510-4520.
[24] RUSSAKOVSKY O, DENG J, SU H, et al. Imagenet large scale visual recognition challenge[J]. Int J Comput Vision, 2015, 115(3): 211-252. doi: 10.1007/s11263-015-0816-y