• 《中国科学引文数据库(CSCD)》来源期刊
  • 中国科技期刊引证报告(核心版)期刊
  • 《中文核心期刊要目总览》核心期刊
  • RCCSE中国核心学术期刊

基于注意力残差机制的细粒度番茄病害识别

胡志伟, 杨华, 黄济民, 谢倩倩

胡志伟, 杨华, 黄济民, 等. 基于注意力残差机制的细粒度番茄病害识别[J]. 华南农业大学学报, 2019, 40(6): 124-132. DOI: 10.7671/j.issn.1001-411X.201812048
引用本文: 胡志伟, 杨华, 黄济民, 等. 基于注意力残差机制的细粒度番茄病害识别[J]. 华南农业大学学报, 2019, 40(6): 124-132. DOI: 10.7671/j.issn.1001-411X.201812048
HU Zhiwei, YANG Hua, HUANG Jiming, et al. Fine-grained tomato disease recognition based on attention residual mechanism[J]. Journal of South China Agricultural University, 2019, 40(6): 124-132. DOI: 10.7671/j.issn.1001-411X.201812048
Citation: HU Zhiwei, YANG Hua, HUANG Jiming, et al. Fine-grained tomato disease recognition based on attention residual mechanism[J]. Journal of South China Agricultural University, 2019, 40(6): 124-132. DOI: 10.7671/j.issn.1001-411X.201812048

基于注意力残差机制的细粒度番茄病害识别

基金项目: 国家自然科学基金 (31671571);国家自然科学基金青年基金 (21803037)
详细信息
    作者简介:

    胡志伟 (1992—),男,硕士,E-mail: zhiweihu@whu.edu.cn

    通讯作者:

    杨 华 (1973—),男,副教授,博士,E-mail: yanghua@sxau.edu.cn

  • 中图分类号: S436.36; TP391.4

Fine-grained tomato disease recognition based on attention residual mechanism

  • 摘要:
    目的 

    解决温室环境下细粒度番茄病害识别方法不足问题。

    方法 

    以早、晚期5种番茄病害叶片为研究对象,提出一种基于注意力与残差思想相结合的新型卷积神经网络模型ARNet。通过引入多层注意力模块,层次化抽取病害分类信息,解决早期病害部位分散、特征难以提取难题;为避免网络训练出现退化现象,构建残差模块有效融合高低阶特征,同时引入数据扩充技术以防止模型过拟合。

    结果 

    对44 295张早、晚期病害叶片数据集进行模型训练与测试的结果表明,与VGG16等现有模型相比,ARNet具有更好的分类表现,其平均识别准确率达到88.2%,显著高于其他模型。ARNet对早期病害识别准确率明显优于晚期病害,验证了注意力机制在提取细微区域特征上的有效性,且在训练过程中未发生过度抖动的状况。

    结论 

    本文提出的模型具有较强鲁棒性和较高稳定性,在实际应用中可为细粒度番茄病害智能诊断提供参考。

    Abstract:
    Objective 

    To solve the insufficient identification of fine-grained tomato diseases in greenhouse.

    Method 

    Taking tomato leaves with five early or late diseases as research objects, we proposed a new convolutional neural network model ARNet based on the combination of attention and residual thought. A multi-layered attention module was introduced to solve the problem of early disease location dispersion and the difficulty of feature extraction by extracting hierarchically disease classification information. In order to avoid the degradation of network training, we constructed a residual module to effectively integrate high- and low-order features. Meantime, we introduced the data expansion technology to prevent model over-fitting.

    Result 

    Model training and testing results of early and late disease leaf datasets with 44 295 pictures showed that ARNet has better classification performance with an average recognition accuracy of 88.2%, which was significantly higher than those of other existing models. In addition, the identification accuracy of ARNet for early disease was significantly better than that for late disease, which verified the effectiveness of attention mechanism in extracting fine region features, and there was no excessive jitter during training process.

    Conclusion 

    This model proposed in this paper has strong robustness and high stability, and can provide a reference for intelligent diagnosis of fine-grained tomato diseases in practical application.

  • 图  1   ARNet模型结构图

    Figure  1.   The structure of ARNet model

    图  2   RCB模块引入前后对照操作

    Figure  2.   The control operations before and after introduction of RCB module

    图  3   番茄病害初始数据集分布

    1:早期白粉病;2:晚期白粉病;3:早期早疫病;4:晚期早疫病;5:早期叶霉病;6:晚期叶霉病;7:早期斑枯病;8:晚期斑枯病;9:早期黄化曲叶病;10:晚期黄化曲叶病

    Figure  3.   The distribution of initial dataset for tomato disease

    1: Early powdery mildew; 2: Later powdery mildew; 3: Early early blight; 4: Later early blight; 5: Early leaf frost disease; 6: Later leaf frost disease; 7: Early spot blight; 8: Later spot blight; 9: Early yellow flower curly leaf disease; 10: Later yellow flower curly leaf disease

    图  4   部分数据预处理前后

    Figure  4.   The partial data before and after preprocessing

    图  5   各层最后ACB模块热力图输出

    Figure  5.   The thermodynamic diagram output of final ACB module at each layer

    图  6   不同模型迭代过程中验证集准确率与损失函数变化

    Figure  6.   The changes of verification set accuracy and loss function in the iterative process of different models

    图  7   3种模型测试集混淆矩阵

    行号1~10分别表示早期白粉病、晚期白粉病、早期早疫病、晚期早疫病、早期叶霜病、晚期叶霜病、早期斑枯病、晚期斑枯病、早期黄化曲叶病以及晚期黄化曲叶病,列号1~10类别与其对应对应编号的行号类别一致

    Figure  7.   The confusion matrixes of test sets of three models

    Line numbers 1 to 10 indicate early powdery mildew, later powdery mildew, early early blight, later early blight, early leaf frost disease, later leaf frost disease, early spot blight, later spot blight, early yellow flower curl leaf disease, and in the case of later yellow flower curl leaf disease, the column numbers 1 to 10 are consistent with the row number category corresponding to the corresponding number

    表  1   不同模型对番茄病害的分类准确率1)

    Table  1   Classification accuracy of tomato diseases by different models

      病害 Disease 时期 Stage VGG16 InceptionV3 Xception MobileNetV2 ResNet34 ARNet
    白粉病 Powdery mildew 早期 Early 0.176 0.191 0.197 0.521 0.521 0.638
    晚期 Late $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.996}}$ 0.978 0.972 0.939 0.967 0.967
    早疫病 Early blight 早期 Early 0.647 0.571 0.541 0.797 0.872 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.940}}$
    晚期 Late $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.909}}$ 0.683 0.706 0.794 0.833 0.865
    叶霜病 Leaf frost disease 早期 Early 0.734 0.814 0.824 0.830 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.920}}$ $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.920}}$
    晚期 Late 0.773 0.432 0.492 0.589 0.627 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.843}}$
    斑枯病 Spot blight 早期 Early 0.573 0.300 0.496 0.712 0.838 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.888}}$
    晚期 Late 0.938 0.913 0.825 0.877 0.931 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.948}}$
    黄化曲叶病
    Yellow flower Curly leaf disease
    早期 Early 0.715 0.693 0.713 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.830}}$ $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.830}}$ 0.826
    晚期 Late $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.911}}$ 0.868 0.855 0.804 0.856 0.887
    平均值 Average 0.817 0.753 0.758 0.807 0.850 $ \setlength{\fboxsep}{0.1cm} \fbox{\bf {0.882}}$
     1)加框数据表示对应列模型在对应行病害中的最佳准确率
     1)Framed data indicated the best accuracy of corresponding column model in the corresponding row disease
    下载: 导出CSV
  • [1]

    BAI X, LI X, FU Z, et al. A fuzzy clustering segmentation method based on neighborhood grayscale information for defining cucumber leaf spot disease images[J]. Comput Electron Agric, 2017, 136: 157-165. doi: 10.1016/j.compag.2017.03.004

    [2] 王翔宇, 温皓杰, 李鑫星, 等. 农业主要病害检测与预警技术研究进展分析[J]. 农业机械学报, 2016, 47(9): 266-277. doi: 10.6041/j.issn.1000-1298.2016.09.037
    [3] 龙满生, 欧阳春娟, 刘欢, 等. 基于卷积神经网络与迁移学习的油茶病害图像识别[J]. 农业工程学报, 2018, 34(18): 194-201. doi: 10.11975/j.issn.1002-6819.2018.18.024
    [4]

    MOKHTAR U, ALI M A S, HASSENIAN A E, et al. Tomato leaves diseases detection approach based on support vector machines[C]//IEEE. Computer Engineering Conference (ICENCO). Egypt: IEEE, 2015: 246-250.

    [5]

    XIE C, SHAO Y, LI X, et al. Detection of early blight and late blight diseases on tomato leaves using hyperspectral imaging[J]. Sci Rep, 2015, 5: 16564. doi: 10.1038/srep16564

    [6] 柴阿丽, 李宝聚, 石延霞, 等. 基于计算机视觉技术的番茄叶部病害识别[J]. 园艺学报, 2010, 37(9): 1423-1430.
    [7]

    AMARA J, BOUAZIZ B, ALGERGAWY A. A deep learning based approach for banana leaf diseases classification[M]// MITSCHANG B. Lecture Notes in Informatics (LNI). Bonn : Gesellschaft Für Informatik, 2017: 79-88.

    [8] 马浚诚, 杜克明, 郑飞翔, 等. 基于卷积神经网络的温室黄瓜病害识别系统[J]. 农业工程学报, 2018, 34(12): 186-192. doi: 10.11975/j.issn.1002-6819.2018.12.022
    [9]

    DURMUS H, GUNES E O, KIRCI M. Disease detection on the leaves of the tomato plants by using deep learning[C]//IEEE. 2017 6th International Conference on Agro-Geoinformatics. Fairfax VA: IEEE, 2017: 1-5.

    [10]

    FUENTES A, YOON S, KIM S C, et al. A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition[J]. Sensors, 2017, 17(9): 2022. doi: 10.3390/s17092022

    [11]

    BRAHIMI M, BOUKHALFA K, MOUSSAOUI A. Deep learning for tomato diseases: Classification and symptoms visualization[J]. Appl Artif Intell, 2017, 31(4): 299-315.

    [12]

    VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Curran Associates. Advances in Neural Information Processing Systems(NIPS). New York: Curran Associates, 2017: 5998-6008.

    [13]

    LETARTE G, PARADIS F, GIGUERE P, et al. Importance of self-attention for sentiment analysis[C]// Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP. Brussels: Association for Computational Linguistics. 2018: 267-275.

    [14]

    YU C, WANG J, PENG C, et al. BiSeNet: Bilateral segmentation network for real-time semantic segmentation [C]//Springer. Proceedings of the European Conference on Computer Vision (ECCV). Munich: Springer, 2018: 325-341.

    [15]

    WANG F, JIANG M, QIAN C, et al. Residual attention network for image classification[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 3156-3164.

    [16]

    FU J, ZHENG H, MEI T. Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 4476-4484.

    [17]

    HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.

    [18]

    HE K, ZHANG X, REN S, et al. Identity mappings in deep residual networks[C]//Springer. Proceedings of the European Conference on Computer Vision (ECCV). Amsterdam : Springer, 2016: 630-645.

    [19]

    ZHANG K, SUN M, HAN T X, et al. Residual networks of residual networks: Multilevel residual networks[J]. IEEE T Circ Syst Vid, 2018, 28(6): 1303-1314. doi: 10.1109/TCSVT.2017.2654543

    [20] 孙俊, 谭文军, 毛罕平, 等. 基于改进卷积神经网络的多种植物叶片病害识别[J]. 农业工程学报, 2017, 33(19): 209-215. doi: 10.11975/j.issn.1002-6819.2017.19.027
    [21]

    SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 2818-2826.

    [22]

    CHOLLET F. Xception: Deep Learning with Depthwise Separable Convolutions[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 1800-1807.

    [23]

    SANDLER M, HOWARD A, ZHU M, et al. MobileNetV2: Inverted residuals and linear bottlenecks[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 4510-4520.

    [24]

    RUSSAKOVSKY O, DENG J, SU H, et al. Imagenet large scale visual recognition challenge[J]. Int J Comput Vision, 2015, 115(3): 211-252. doi: 10.1007/s11263-015-0816-y

图(7)  /  表(1)
计量
  • 文章访问数:  1642
  • HTML全文浏览量:  12
  • PDF下载量:  2170
  • 被引次数: 0
出版历程
  • 收稿日期:  2018-12-24
  • 网络出版日期:  2023-05-17
  • 刊出日期:  2019-11-09

目录

    /

    返回文章
    返回