Recognition of Huanglongbing symptom based on deep convolutional neural network
-
摘要:目的
探究深度学习在柑橘 Citrus spp.黄龙病症状识别上的可行性,并评估识别器的识别准确率。
方法以黄龙病/非黄龙病引起的发病叶片图像及健康叶片图像为训练素材,基于卷积神经网络及迁移学习技术构建二类识别器(I-2-C和M-2-C)和八类识别器(I-8-C和M-8-C)。
结果M-8-C模型的整体识别表现最优,对所有图像的识别准确率为93.7%,表明构建的神经网络识别器能有效辨别柑橘黄龙病症状;I-8-C和M-8-C对所有类型图像的平均F1分值分别为77.9%和88.4%,高于I-2-C(56.3%)和M-2-C(52.5%),表明症状细分有利于提高模型的识别能力。同时M-8-C比I-8-C略高的平均F1分值表明基于MobileNetV1结构的八类识别器识别表现略优于基于InceptionV3的八类识别器。基于M-8-C改进的识别器M-8f-C能够转移到智能手机上,在田间测试中取得较好的识别表现。
结论基于深度学习和迁移学习开发的识别器对黄龙病单叶症状具有较好的识别效果。
Abstract:ObjectiveTo explore the capability of deploying deep learning to the detection of Huanglongbing (HLB) symptom inCitrus spp., and evaluate the classification accuracies of the classifiers.
MethodTwo-class classifiers(I-2-C and M-2-C) and eight-class classifiers(I-8-C and M-8-C) were constructed using images of diseased leaves caused by HLB/non-HLB and healthy leaves based on convolutional neural networks and transfer learning.
ResultThe overall classification performance of M-8-C stood out in all classifiers with accuracy of 93.7%, implying great capability in deep convolutional neural networks for classifying HLB symptoms. The mean F1 socres of I-8-C and M-8-C were 77.9% and 88.4% respectively, which were higher than those of I-2-C(56.3%) and M-2-C(52.5%). This indicated that subtyping symptoms could help improve the recognition ability of models. The slightly higher mean F1 score of M-8-C compared with I-8-C indicated that the eight-class model based on MobileNetV1 had better performance than the one based on InceptionV3. An optimized model, namely M-8f-C, was developed based on M-8-C and was successfully mounted on mobile phone. The field tests showed that M-8f-C was of decent performance under field conditions.
ConclusionClassifier based on deep learning and transfer learning has high accuracy for recognizing HLB symptom leaves.
-
-
图 1 本研究收集的8种类别柑橘叶片图像示例
A:健康;B:黄龙病−斑驳;C:黄龙病−缺锌状;D:黄龙病−叶脉黄化;E:黄龙病−均匀黄化;F:非黄龙病−缺锌状;G:非黄龙病−叶脉黄化;H:非黄龙病−均匀黄化
Figure 1. Examples of eight classes of citrus leaves used in this study
A: Healthy; B: HLB-mottle; C: HLB-zinc deficiency; D: HLB-vein yellowing; E: HLB-uniform yellowing; F: Non-HLB-zinc deficiency; G: Non-HLB-vein yellowing; H: Non-HLB-uniform yellowing
表 1 使用MobileNetV1网络结构的八类识别器的测试结果
Table 1 Test result of eight-class classifier using MobileNetV1 network architecture
图片类型
Image
type测试集
图片数
No. of
images in test set训练集
图片数
No. of images
in training set真阳性
率/%
True positive
rate假阴性
率/%
False negative
rate真阴性
率/%
True negative
rate假阳性
率/%
False positive
rate准确率/%
Accuracy平均F1
分值/%
Mean F1
score黄龙病
识别率/%
HLB detection
rate非黄龙病
识别率/%
Non-HLB
detection rate黄龙病−斑驳 HLB-mottle 50 514 84.0 16.0 77.4 6.0 91.0 87.9 94.0 6.0 黄龙病−缺锌状
HLB-zinc deficiency50 120 86.0 14.0 77.1 2.6 94.6 86.1 100.0 0.0 黄龙病−叶脉黄化
HLB-vein yellowing50 115 74.0 26.0 78.9 3.4 92.1 85.9 90.0 10.0 黄龙病−均匀黄化
HLB-uniform yellowing50 131 44.0 56.0 83.1 4.3 87.5 89.7 86.0 14.0 健康 Healthy 50 233 92.0 8.0 76.3 2.0 96.1 80.6 6.0 94.0 非黄龙病−缺锌状
Non-HLB-zinc deficiency50 123 98.0 2.0 75.4 1.4 97.6 95.3 2.0 98.0 非黄龙病−叶脉黄化
Non-HLB-vein yellowing50 254 84.0 16.0 77.4 2.6 94.3 91.3 6.0 94.0 非黄龙病−均匀黄化
Non-HLB-uniform yellowing50 177 64.0 36.0 80.3 1.7 92.4 90.9 26.0 74.0 所有黄龙病图像
Total HLB images200 880 72.0 44.0 79.1 4.1 91.3 87.4 85.5 7.5 所有非黄龙病图像
Total non-HLB images200 787 84.5 15.5 77.4 1.9 95.1 89.5 10.0 90.0 所有图像 Total images 400 1 667 78.3 21.8 78.3 3.0 93.7 88.4 表 2 使用InceptionV3网络结构的八类识别器的测试结果
Table 2 Test result of eight-class classifier using InceptionV3 network architecture
图片类型
Image
type测试集
图片数
No. of
images in test set训练集
图片数
No. of images
in training set真阳性
率/%
True positive
rate假阴性
率/%
False negative
rate真阴性
率/%
True negative
rate假阳性
率/%
False positive
rate准确率/%
Accuracy平均F1
分值/%
Mean F1
score黄龙病
识别率/%
HLB detection
rate非黄龙病
识别率/%
Non-HLB
detection rate黄龙病−斑驳 HLB-mottle 50 514 56.0 42.0 66.6 6.0 85.7 84.3 82.0 16.0 黄龙病−缺锌状
HLB-zinc deficiency50 120 84.0 16.0 62.6 10.0 85.4 74.7 94.0 6.0 黄龙病−叶脉黄化
HLB-vein yellowing50 115 68.0 32.0 64.9 4.0 89.2 50.6 78.0 22.0 黄龙病−均匀黄化
HLB-uniform yellowing50 131 14.0 86.0 72.6 5.7 80.1 89.3 88.0 12.0 健康 Healthy 50 233 62.0 38.0 65.7 4.9 87.4 74.3 16.0 84.0 非黄龙病−缺锌状
Non-HLB-zinc deficiency50 123 92.0 8.0 61.4 2.6 94.7 94.2 0.0 100.0 非黄龙病−叶脉黄化
Non-HLB-vein yellowing50 254 84.0 18.0 62.6 4.6 90.7 83.2 10.0 92.0 非黄龙病−均匀黄化
Non-HLB-uniform yellowing50 177 62.0 38.0 65.7 2.0 90.4 72.7 34.0 66.0 所有黄龙病图像
Total HLB images200 880 55.5 44.0 66.6 6.4 85.1 74.7 85.5 14.0 所有非黄龙病图像
Total non-HLB images200 787 75.0 25.5 63.9 3.5 90.8 81.1 15.0 85.5 所有图像 Total images 400 1 667 65.3 34.3 65.3 5.0 88.0 77.9 表 3 使用MobileNetV1网络结构的二类识别器的测试结果
Table 3 Test result of two-class classifier using MobileNetV1 network architecture
图片类型
Image
type测试集
图片数
No. of
images in test set训练集
图片数
No. of
images in training set真阳性
率/%
True positive
rate假阴性
率/%
False negative
rate真阴性
率/%
True negative
rate假阳性
率/%
False positive
rate准确率/%
Accuracy平均F1
分值/%
Mean F1
score黄龙病
识别率/%
HLB detection
rate非黄龙病
识别率/%
Non-HLB
detection rate所有黄龙病图像
Total HLB images200 880 96.0 4.0 22.0 78.0 59.0 70.1 96.0 4.0 所有非黄龙病图像
Total non-HLB images200 787 22.0 78.0 96.0 4.0 59.0 34.9 78.0 22.0 所有图像 Total images 400 1 667 59.0 41.0 59.0 41.0 59.0 52.5 表 4 使用InceptionV3网络结构的二类识别器的测试结果
Table 4 Test result of two-class classifier using InceptionV3 network architecture
图片类型
Image
type测试集
图片数
No. of
images in test set训练集
图片数
No. of
images in training set真阳性
率/%
True positive
rate假阴性
率/%
False negative
rate真阴性
率/%
True negative
rate假阳性
率/%
False positive
rate准确率/%
Accuracy平均F1
分值/%
Mean F1
score黄龙病
识别率/%
HLB detection
rate非黄龙病
识别率/%
Non-HLB
detection rate所有黄龙病图像
Total HLB images200 880 84.0 16.0 34.0 66.0 59.0 67.2 84.0 16.0 所有非黄龙病图像
Total non-HLB images200 787 34.0 66.0 84.0 16.0 59.0 45.3 66.0 34.0 所有图像 Total images 400 1 667 59.0 41.0 59.0 41.0 59.0 56.3 表 5 识别器的田间测试结果
Table 5 Field test result of classifier
测试平台
Test platform图像背景
Image Background图片类别
Image
type测试集
图片数
No. of
images in test set训练集
图片数
No. of
images in training set真阳性
率/%
True positive
rate假阴性
率/%
False negative
rate真阴性
率/%
True negative
rate假阳性
率/%
False positive
rate准确率/%
Accuracy平均F1
分值/%
Mean F1
score手机
Mobile phone树上
On the tree黄龙病
HLB150 4 280 100 0 68.7 29.3 85.2 87.2 非黄龙病
Non-HLB147 4 534 68.7 29.3 100 0 85.2 82.4 摄影布
Photographic backdrop黄龙病
HLB150 4 280 56.0 44.0 91.3 8.7 73.7 68.0 非黄龙病
Non-HLB151 4 534 91.3 8.7 56.0 44.0 73.7 77.6 计算机
Computer树上
On the tree黄龙病
HLB149 4 280 76.5 23.5 99.4 0.6 88.3 86.4 非黄龙病
Non-HLB159 4 534 99.4 0.6 76.5 23.5 88.3 89.8 摄影布
Photographic backdrop黄龙病
HLB157 4 280 98.7 1.3 85.7 13.2 92.4 93.1 非黄龙病
Non-HLB147 4 534 85.7 14.3 98.7 1.3 92.4 91.6 -
[1] 林孔湘. 柑桔黄梢(黄龙)病研究Ⅰ: 病情调查[J]. 植物病理学报, 1956, 2(1): 1-11. [2] DENG X, LAN Y, HONG T, et al. Citrus greening detection using visible spectrum imaging and C-SVC[J]. Comput Electron Agr, 2016, 130(2016): 177-183.
[3] GARCIA-RUIZ F, SANKARAN S, MAJA J M, et al. Comparison of two aerial imaging platforms for identification of Huanglongbing-infected citrus trees[J]. Comput Electron Agr, 2013, 91: 106-115. doi: 10.1016/j.compag.2012.12.002
[4] MISHRA A, KARIMI D, EHSANI R, et al. Evaluation of an active optical sensor for detection of Huanglongbing (HLB) disease[J]. Biosyst Eng, 2011, 110(3): 302-309. doi: 10.1016/j.biosystemseng.2011.09.003
[5] MISHRA A, KARIMI D, EHSANI R, et al. Identification of citrus greening (Hlb) using a vis-nir spectroscopy technique[J]. T ASABE, 2012, 55(2): 711-720. doi: 10.13031/2013.41369
[6] SANKARAN S, MISHRA A, MAJA J M, et al. Visible-near infrared spectroscopy for detection of Huanglongbing in citrus orchards[J]. Comput Electron Agr, 2011, 77(2): 127-134. doi: 10.1016/j.compag.2011.03.004
[7] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Commun ACM, 2017, 60(6): 84-90. doi: 10.1145/3065386
[8] ZHANG Y C, KAGEN A C. Machine learning interface for medical image analysis[J]. J Digit Imaging, 2017, 30(5): 615-621. doi: 10.1007/s10278-016-9910-0
[9] ESTEVA A, KUPREL B, NOVOA R A, et al. Dermatologist-level classification of skin cancer with deep neural networks[J]. Nature, 2017, 542(7639): 115-118. doi: 10.1038/nature21056
[10] LEE H, KIM K G. Applying deep learning in medical images: The case of bone age estimation[J]. Healthc Inform Res, 2018, 24(1): 86-92. doi: 10.4258/hir.2018.24.1.86
[11] LITJENS G, KOOI T, BEJNORDI B E, et al. A survey on deep learning in medical image analysis[J]. Med Image Anal, 2017, 42: 60-88. doi: 10.1016/j.media.2017.07.005
[12] GHOSAL S, BLYSTONE D, SINGH A K, et al. An explainable deep machine vision framework for plant stress phenotyping[J]. Proc Natl Acad Sci USA, 2018, 115(18): 4613-4618. doi: 10.1073/pnas.1716999115
[13] TAPAS A. Transfer learning for image classification and plant phenotyping[J]. Int J Res Appl Sci Eng Technol, 2016, 5(11): 2664-2668.
[14] SLADOJEVIC S, ARSENOVIC M, CULIBRK A A D, et al. Deep neural networks based recognition of plant diseases by leaf image classification[J]. Comput Intel Neurosc, 2016: 3289801.
[15] ZHANG K, WU Q, LIU A, et al. Can deep learning identify tomato leaf disease?[J]. Advances in Multimedia, 2018, 2018: 1-10.
[16] RAMCHARAN A, BARANOWSKI K, MCCLOSKEY P, et al. Deep learning for image-based cassava disease detection[J]. Front Plant Sci, 2017, 8: 1-7.
[17] CHOI D, LEE W, SCHUELLER J K, et al. A precise fruit inspection system for Huanglongbing and other common citrus defects using GPU and deep learning technologies[C]// ISPA. Proceedings of 13th International Conference on Precision Agriculture, St Louis, Missouri: ISPA, 2016, 1-6.
[18] MOHANTY S P, HUGHES D P, SALATHE M. Using deep learning for image-based plant disease detection[J]. Front Plant Sci, 2016, 7: 1-7.
[19] LI W, HARTUNG J S, LEVY L. Quantitative real-time PCR for detection and identification of Candidatus Liberibacter species associated with citrus huanglongbing[J]. J Microbiol Meth, 2006, 66(1): 104-115. doi: 10.1016/j.mimet.2005.10.018
[20] FAWCETT T. An introduction to ROC analysis[J]. Pattern Recogn Lett, 2006, 27(8): 861-874. doi: 10.1016/j.patrec.2005.10.010
[21] DENG X, LAN Y, XING X, et al. Citrus huanglongbing detection based on image feature extraction and two-stage back propagation neural network modeling[J]. Int J Agr Biol Eng, 2016, 9(6): 20-26.
[22] 李俭川, 秦国军, 温熙森, 等. 神经网络学习算法的过拟合问题及解决方法[J]. 振动、测试与诊断, 2002, 22(4): 260-264. doi: 10.3969/j.issn.1004-6801.2002.04.003 [23] 陶砾, 杨朔, 杨威. 深度学习的模型搭建及过拟合问题的研究[J]. 计算机时代, 2018(2): 14-21. [24] REINKING O A. Diseases of economic plants in southern China[J]. Philipp Agric, 1919, 8: 109-135.
[25] TU C. Notes on disease of economic plants in South China[J]. Lingnan Science Journal, 1932, 11: 489-504.
[26] 吴定尧. 柑橘黄龙病及综合防治[M]. 广州: 广东科技出版社. 2010: 12-20.